Signal processing device

ABSTRACT

The present invention relates to a signal processing device which learns operations made by a user without the user knowing, and which can perform processing optimal to the user based on the learning results. At a learning unit  22,  operating signals supplied according to user operations are monitored, and judgment is made whether or not these can be used for learning. In the event that the operating signals are learning operating signals which can be used for learning, the learning unit  22  learns a correction norm which is the norm for correcting input signals, based on the learning operating signals. On the other hand, at a correcting unit  21,  post-correction signals, wherein the input signals are corrected based on the correction norm obtained by learning, are output as output signals. The present invention can be applied to an NR (Noise Reduction) circuit which removes noise.

TECHNICAL FIELD

[0001] The present invention relates to a signal processing device, andparticularly relates to a signal processing device which, for example,changes the contents of processing or the structure of processing byoperations made by the user, so as to perform optimal processing for theuser.

BACKGROUND ART

[0002] For example, with conventional NR (Noise Reduction) circuits, inthe event that a user operates an adjustment knob thereof and sets it ata certain position, noise removal processing corresponding to theposition of the knob is performed.

[0003] Now, the S/N (Signal to Noise Ratio) and frequency properties andthe like of signals input to the NR circuit are not necessarily alwaysconstant, rather, it is common for these to change. In the event thatthe S/N, frequency properties, etc., of the signals input to the NRcircuit change, suitable processing is not necessarily performed withregard to the signals input to the NR circuit in the noise removalprocessing corresponding to the position where the user has set theknob, and accordingly, the user needs to frequently operate the knob sothat suitable noise removal processing is performed for him/herself,which has been troublesome.

DISCLOSURE OF INVENTION

[0004] The present invention has been made in light of such, andaccordingly, it is an object thereof to change the contents ofprocessing or the structure of processing by operations made by theuser, so as to perform optimal processing for the user.

[0005] A signal processing device according to the present inventioncomprises: signal processing means for signal processing of inputsignals; and output means for outputting the signal processing resultsof the signal processing means; wherein the structure of processing ofthe signal processing means is changed based on operating signalssupplied according to user operations.

[0006] The signals processing means may comprise: characteristicsdetecting means for detecting characteristics from the input signals;processing deciding means for deciding contents of processing as to theinput signals, based on characteristics detected by the characteristicsdetecting means; and processing executing means for executing processingto the input signals, following the contents of processing decided bythe processing deciding means; and the structure of processing of atleast one of the characteristics detecting means, the processingdeciding means, or the processing executing means, may be changed basedon the operating signals.

[0007] The output means may comprise presenting means for presenting thesignal processing results of the signals processing means.

[0008] The structure of processing of the characteristics detectingmeans may be changed based on the operating signals.

[0009] The operating signals may be signals which specify, of aplurality of types of characteristics, a predetermined number of typesof characteristics; and the characteristics detecting means may changethe structure of processing thereof so as to detect the predeterminednumber of types of characteristics.

[0010] The characteristics detecting means may detect the predeterminednumber of types of characteristics from the input signals; and theprocessing deciding means may decide the contents of processing at theprocessing executing means as to the input signals, based on thepredetermined number of types of characteristics detected from the inputsignals by the characteristics detecting means.

[0011] The input signals may be image signals; the processing decidingmeans may decide whether or not to output the input signals withoutchange as the contents of processing as to the input signals, based onthe predetermined number of types of characteristics detected from theinput signals by the characteristics detecting means; and theprocessing-executing means may detect a telop within image signals whichare the input signals, by selectively outputting the input signalsfollowing the decision of the processing deciding means.

[0012] The characteristics detecting means may change the structure ofprocessing thereof so as to detect new types of characteristicsdiffering from characteristics which are provided beforehand, based onthe operating signals.

[0013] The characteristics detecting means may detect the dynamic rangeof the characteristics prepared beforehand, maximum value, centralvalue, minimum value, sum, dispersion, the number of input signals withthe characteristics having a value greater than a threshold value, orand linear combination of the characteristics, as the new types ofcharacteristics.

[0014] The structure of processing of the processing deciding means maybe changed based on the operating signals.

[0015] The processing deciding means may store acharacteristics/processing correlation relationship which is acorrelation relationship between the values of the characteristics, andthe contents of processing as to the input signals havingcharacteristics of those values; and may decide the contents ofprocessing correlated to the values of characteristics detected from theinput signals as the contents of processing as to the input signals, inthe characteristics/processing correlation relationship.

[0016] The processing deciding means may change the structure ofprocessing thereof by changing the contents of processing in thecharacteristics/processing correlation relationship, based on theoperating signals.

[0017] In the characteristics/processing correlation relationship, twocontents of processing may exist as to the input signals; these may beprocessing for outputting output signals of a first value, andprocessing for outputting output signals of a second value; and theprocessing executing means may binarize the input signals into the firstand second values, following the decision of the processing decidingmeans.

[0018] The structure of processing of the processing executing means maybe changed based on the operating signals.

[0019] The processing executing means may comprise: tutor datagenerating means for generating tutor data from predetermined learningdata; student data generating means for generating student data from thelearning data; learning means for learning a prediction value of thetutor data obtained by linear combination of the student data and apredetermined prediction coefficient, and the prediction coefficientwhich minimizes the error with the tutor data; and output signalsgenerating means for generating output signals by performing linearcombination of the input signals and a prediction coefficient obtainedby the learning means.

[0020] The learning means may learn the prediction coefficient byleast-N′th-power error which statistically minimizes an N′th-power errorwhich is the error to the N′th power; and may change the structure ofprocessing thereof by changing the N′th power of the error, based on theoperating signals.

[0021] The learning means may change the N′th power of error based onthe learning signals, by employing the product of a square-error as theN′th power of error and a weight according to the operating signals.

[0022] The learning means may employ, as the N′th power error using anN′th power corresponding to the operating signals, the product of asquare-error and a N−2′th error of a prediction value of the tutor datacomputed using the prediction coefficient obtained by least-N′th-error.

[0023] The learning means may learn the prediction coefficient for eachcontents of processing which the processing deciding means decide; andthe output generating means may generate the output signals by linearcombination of the input signals, and the prediction coefficientcorresponding to the contents of processing which the processingdeciding means decide based on the input signals.

[0024] The signals processing means may comprise: judging means formonitoring the operating signals judging whether or not usable forlearning; learning means for learning a correction norm which is a normfor correcting the input signals, based on learning operating signalswhich are the operating signals usable for learning; and correctingmeans for correcting the input signals based on the correction normobtained by the learning, and outputting the post-correction signalsthereof as output signals.

[0025] The signals processing means may comprise: tutor data generatingmeans for generating tutor data from predetermined learning data;student data generating means for generating student data from thelearning data; learning means for learning a prediction value of thetutor data obtained by linear combination of the student data and apredetermined prediction coefficient, and the prediction coefficientwhich minimizes the error with the tutor data; and output signalsgenerating means for generating output signals by performing linearcombination of the input signals and a prediction coefficient obtainedby the learning means.

[0026] The signal processing means may learn the prediction coefficientby least-N′th-power error which statistically minimizes an N′th-powererror which is the error to the N′th power; and may change the structureof processing thereof by changing the N′th power of the error, based onthe operating signals.

[0027] The learning means may change the N′th power of error based onthe operating signals, by employing the product of a square-error as theN′th power of error and a weight according to the operating signals.

[0028] The learning means may employ, as the N′th power error using anN′th power corresponding to the operating signals, the product of asquare-error and a N−2′th error of a prediction value of the tutor datacomputed using the prediction coefficient obtained by least-N′th-error.

[0029] A signal processing method according to the present inventioncomprises: a signal processing step for signals processing of inputsignals; and an output step for outputting the signal processing resultsof the processing in the signal processing step; wherein the structureof processing in the signal processing step is changed based onoperating signals supplied according to user operations.

[0030] A program of a recording medium according to the presentinvention comprises: a signal processing control step for controllingsignal processing of input signals; and an output control step forcontrolling output of the signal processing results of the processing inthe signal processing control step; wherein the structure of processingin the signal processing control step is changed based on operatingsignals supplied according to user operations.

[0031] The program according to the present invention causes a computerto execute: a signal processing control step for controlling signalprocessing of input signals; and an output control step for controllingoutput of the signal processing results of the processing in the signalprocessing control step; wherein the structure of processing in thesignal processing control step is changed based on operating signalssupplied according to user operations.

[0032] With the present invention, input signals are subjected to signalprocessing, signal processing results are output, and the structure ofprocessing of signal processing is changed based on operation signalssupplied according to user operations.

BRIEF DESCRIPTION OF THE DRAWINGS

[0033]FIG. 1 is a diagram illustrating an optimizing device to which thepresent invention has been applied.

[0034]FIG. 2 is a block diagram illustrating a configuration example ofan embodiment of an optimizing device to which the present invention hasbeen applied.

[0035]FIG. 3 is a flowchart describing optimizing processing with theoptimizing device shown in FIG. 2.

[0036]FIG. 4 is a block diagram illustrating a configuration example ofan embodiment of an NR circuit using an optimizing device.

[0037]FIG. 5A is a waveform diagram representing input signals.

[0038]FIG. 5B is a waveform diagram representing input reliability.

[0039]FIG. 6 is a flowchart describing correction processing with an NRcircuit.

[0040]FIG. 7 is a flowchart describing correction parameter computingprocessing with an NR circuit.

[0041]FIG. 8 is a flowchart describing control data learning processingwith an NR circuit.

[0042]FIG. 9A through FIG. 9C are diagrams describing control datalearning processing.

[0043]FIG. 10 is a block diagram illustrating a configuration example ofanother embodiment of an NR circuit using an optimizing device.

[0044]FIG. 11 is a diagram illustrating pixels multiplied with parametercontrol data.

[0045]FIG. 12 is a flowchart describing correction parameter computingprocessing with an NR circuit.

[0046]FIG. 13 is a flowchart describing control data learning processingwith an NR circuit.

[0047]FIG. 14 is a block diagram illustrating a configuration example ofanother embodiment of an NR circuit using an optimizing device.

[0048]FIG. 15 is a flowchart describing optimizing processing with theoptimizing device shown in FIG. 14.

[0049]FIG. 16 is a block diagram illustrating a configuration example ofan embodiment of an auto-driving device to which the present inventionhas been applied.

[0050]FIG. 17 is a block diagram illustrating a configuration example ofthe processing unit of the optimizing device shown in FIG. 16.

[0051]FIG. 18 is a flowchart describing correction parameter computingprocessing with the optimizing device shown in FIG. 16.

[0052]FIG. 19 is a flowchart describing control data learning processingwith the optimizing device shown in FIG. 16.

[0053]FIG. 20 is a block diagram illustrating another configurationexample of the processing unit of the optimizing device shown in FIG.16.

[0054]FIG. 21 is a diagram illustrating a driving direction output bythe computing unit shown in FIG. 16

[0055]FIG. 22 is a flowchart describing the correction processing by theoptimizing device shown in FIG. 16.

[0056]FIG. 23 is a flowchart describing the correction parameterlearning processing by the optimizing device shown in FIG. 16.

[0057]FIG. 24 is a block diagram illustrating another configurationexample of an auto-driving device to which the present invention hasbeen applied.

[0058]FIG. 25 is a flowchart describing correction parameter computingprocessing by the optimizing device shown in FIG. 24.

[0059]FIG. 26 is a flowchart describing correction parameter learningprocessing by the optimizing device shown in FIG. 24.

[0060]FIG. 27 is a diagram illustrating an example of internalinformation generated by the internal information generating unit shownin FIG. 24.

[0061]FIG. 28 is a diagram illustrating an example of internalinformation generated by the internal information generating unit shownin FIG. 24.

[0062]FIG. 29 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0063]FIG. 30 is a block diagram illustrating a configuration example ofthe learning unit of the optimizing device shown in FIG. 29.

[0064]FIG. 31 is a block diagram illustrating a configuration example ofthe mapping processing unit of the optimizing device shown in FIG. 29.

[0065]FIG. 32 is a diagram describing error between a true value and aprediction value.

[0066]FIG. 33 is a diagram describing the least-N′th-power error method.

[0067]FIG. 34 is a diagram describing weight α_(s).

[0068]FIG. 35 is a flowchart describing image optimizing processing withthe optimizing device shown in FIG. 29.

[0069]FIG. 36 is a diagram illustrating a comparison between aleast-N′th-power norm and a least-square norm.

[0070]FIG. 37 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0071]FIG. 38 is a flowchart describing image optimizing processing withthe optimizing device shown in FIG. 37.

[0072]FIG. 39 is a diagram illustrating an example of internalinformation generated by the internal information generating unit shownin FIG. 37.

[0073]FIG. 40 is a diagram illustrating an example of internalinformation generated by the internal information generating unit shownin FIG. 37.

[0074]FIG. 41 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0075]FIG. 42 is a block diagram illustrating a configuration example ofthe coefficient converting unit of the optimizing device shown in FIG.41.

[0076]FIG. 43 is a block diagram illustrating a configuration example ofa learning device which generates coefficients stored in the coefficientmemory shown in FIG. 41 by learning.

[0077]FIG. 44 is a flowchart describing the coefficient decidingprocessing by the learning device shown in FIG. 43.

[0078]FIG. 45 is a diagram illustrating the configuration of aprediction tap.

[0079]FIG. 46 is a diagram illustrating an example of the distributionof coefficient values corresponding to the tap locations of theprediction tap.

[0080]FIG. 47 is a diagram illustrating the configuration of aprediction tap.

[0081]FIG. 48 is a diagram illustrating an example of the distributionof coefficient values corresponding to the tap locations of theprediction tap.

[0082]FIG. 49 is a diagram illustrating the configuration of aprediction tap.

[0083]FIG. 50 is a diagram illustrating a spring model.

[0084]FIG. 51 is a diagram illustrating a equilibrium model.

[0085]FIG. 52 is a flowchart describing image optimizing processing withthe optimizing device shown in FIG. 41.

[0086]FIG. 53 is a block diagram illustrating a configuration of anotherembodiment of an optimizing device to which the present invention hasbeen applied.

[0087]FIG. 54 is a flowchart describing image optimizing processing withthe optimizing device shown in FIG. 53.

[0088]FIG. 55 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0089]FIG. 56 is a block diagram illustrating a configuration example ofthe features detecting unit shown in FIG. 55.

[0090]FIG. 57 is a block diagram illustrating a configuration example ofthe processing deciding unit shown in FIG. 55.

[0091]FIG. 58 is a block diagram illustrating a configuration example ofthe processing unit shown in FIG. 55.

[0092]FIG. 59 is a flowchart describing telop extraction optimizingprocessing by the optimizing device shown in FIG. 55.

[0093]FIG. 60 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0094]FIG. 61 is a flowchart describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 60.

[0095]FIG. 62 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0096]FIG. 63 is a flowchart describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 62.

[0097]FIG. 64 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 62.

[0098]FIG. 65 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 62.

[0099]FIG. 66 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 62.

[0100]FIG. 67 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 62.

[0101]FIG. 68 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 62.

[0102]FIG. 69 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0103]FIG. 70 is a block diagram illustrating a configuration example ofthe processing deciding unit shown in FIG. 69.

[0104]FIG. 71 is a flowchart describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0105]FIG. 72 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0106]FIG. 73 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0107]FIG. 74 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0108]FIG. 75 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0109]FIG. 76 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0110]FIG. 77 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0111]FIG. 78 is a diagram describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 69.

[0112]FIG. 79 is a diagram describing switching over of features withthe optimizing device shown in FIG. 69.

[0113]FIG. 80 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0114]FIG. 81 is a block diagram illustrating a configuration example ofthe features detecting unit shown in FIG. 80.

[0115]FIG. 82 is a flowchart describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 80.

[0116]FIG. 83 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0117]FIG. 84 is a block diagram illustrating a configuration example ofthe features detecting unit shown in FIG. 80.

[0118]FIG. 85 is a flowchart describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 84.

[0119]FIG. 86 is a diagram describing an alteration contents instructingscreen for features, with the optimizing device shown in FIG. 84.

[0120]FIG. 87 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0121]FIG. 88 is a flowchart describing telop extraction optimizingprocessing with the optimizing device shown in FIG. 87.

[0122]FIG. 89 is a block diagram illustrating a configuration example ofanother embodiment of an optimizing device to which the presentinvention has been applied.

[0123]FIG. 90 is a block diagram illustrating a configuration example ofthe processing deciding unit shown in FIG. 89.

[0124]FIG. 91 is a flowchart describing image optimizing processing withthe optimizing device shown in FIG. 84.

[0125]FIG. 92 is a diagram illustrating an LUT.

[0126]FIG. 93 is a diagram describing the processing contents specifiedfor each of the features on the LUT.

[0127]FIG. 94 is a diagram describing the processing contents specifiedfor each of the features on the LUT.

[0128]FIG. 95 is a diagram describing the processing contents specifiedfor each of the features on the LUT.

[0129]FIG. 96 is a flowchart describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0130]FIG. 97 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0131]FIG. 98 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0132]FIG. 99 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0133]FIG. 100 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0134]FIG. 101 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0135]FIG. 102A is a diagram describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0136]FIG. 102B is a diagram describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0137]FIG. 103A is a diagram describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0138]FIG. 103B is a diagram describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0139]FIG. 104A is a diagram describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0140]FIG. 104B is a diagram describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.91.

[0141]FIG. 105 is a flowchart describing auto LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0142]FIG. 106 is a diagram describing auto LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0143]FIG. 107 is a diagram describing auto LUT changing-processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0144]FIG. 108 is a diagram describing auto LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0145]FIG. 109 is a diagram describing auto LUT changing processing inimage optimizing processing with the optimizing device shown in FIG. 91.

[0146]FIG. 110 is a block diagram illustrating a configuration exampleof another embodiment of an optimizing device to which the presentinvention has been applied.

[0147]FIG. 111 is a block diagram illustrating a configuration exampleof the processing deciding unit shown in FIG. 110.

[0148]FIG. 112 is a flowchart describing image optimizing processingwith the optimizing device shown in FIG. 110.

[0149]FIG. 113 is a flowchart describing manual LUT changing processingin image optimizing processing with the optimizing device shown in FIG.110.

[0150]FIG. 114 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG.110.

[0151]FIG. 115 is a diagram describing manual LUT changing processing inimage optimizing processing with the optimizing device shown in FIG.110.

[0152]FIG. 116 is a block diagram illustrating a configuration exampleof another embodiment of an optimizing device to which the presentinvention has been applied.

[0153]FIG. 117 is a block diagram illustrating a configuration exampleof the processing unit shown in FIG. 116.

[0154]FIG. 118 is a block diagram of a learning device for generatingcoefficient sets stored in the coefficient memory shown in FIG. 116 bylearning.

[0155]FIG. 119 is a block diagram illustrating a configuration exampleof the mapping processing unit shown in FIG. 117.

[0156]FIG. 120 is a flowchart describing learning processing with theoptimizing device shown in FIG. 116.

[0157]FIG. 121 is a flowchart describing mapping processing with theoptimizing device shown in FIG. 116.

[0158]FIG. 122 is a block diagram illustrating a configuration exampleof another embodiment of an optimizing device to which the presentinvention has been applied.

[0159]FIG. 123 is a block diagram illustrating a configuration exampleof the processing unit shown in FIG. 122.

[0160]FIG. 124 is a flowchart describing learning processing by theoptimizing device shown in FIG. 122.

[0161]FIG. 125 is a flowchart describing mapping processing by theoptimizing device shown in FIG. 122

[0162]FIG. 126 is a block diagram illustrating a configuration exampleof another embodiment of an optimizing device to which the presentinvention has been applied.

[0163]FIG. 127 is a block diagram illustrating a configuration exampleof the processing unit shown in FIG. 126.

[0164]FIG. 128 is a block diagram of a learning device for generatingcoefficient sets stored in the coefficient memory shown in FIG. 127 bylearning.

[0165]FIG. 129 is a flowchart describing coefficients decidingprocessing with the learning device shown in FIG. 128.

[0166]FIG. 130 is a flowchart describing image optimizing processingwith the optimizing device shown in FIG. 126.

[0167]FIG. 131 is a block diagram illustrating a configuration exampleof another embodiment of an optimizing device to which the presentinvention has been applied.

[0168]FIG. 132 is a block diagram illustrating a configuration exampleof the processing unit shown in FIG. 131.

[0169]FIG. 133 is a flowchart describing image optimizing processingwith the optimizing device shown in FIG. 131.

[0170]FIG. 134 is a block diagram illustrating a configuration exampleof an embodiment of a computer to which the present invention has beenapplied.

BEST MODE FOR CARRYING OUT THE INVENTION

[0171]FIG. 1 illustrates a configuration example of an embodiment of anoptimizing device to which the present invention has been applied.Following executing predetermined processing with regard to inputsignals (signal processing), the optimizing device outputs signalsobtained as the results of the processing, as output signals. The userexamines (qualitatively evaluates) the output signals, and in the eventthat the output signals do not match personal preferences, inputsoperating signals corresponding to the preferences of the user, to theoptimizing device. The optimizing device changes the contents ofprocessing and structure of processing based on the operating signals,subjects the input signals to the predetermined processing again, andoutputs the output signals. The optimizing device subjects the inputsignals to optimal processing (processing which is optimal for the user)while repeating changing the contents of the processing and structure ofprocessing corresponding to operating signals input by user operations,thereby outputting output signals which are closer to the preferences ofthe user.

[0172]FIG. 2 illustrates a first detailed configuration example of theoptimizing device shown in FIG. 1.

[0173] With this optimizing device 1, processing which is optimal forthe user is performed by learning the operations of the user without theuser knowing. That is, with the optimizing device, operating signalssupplied corresponding to operations of the user are monitored, andjudgment is passed regarding whether or not these can be used forlearning. In the event that the operating signals are learning operatingsignals which can be used for learning, a correction norm which is thenorm for correcting the input signals is learned, based on the learningoperating signals. On the other hand, input signals are corrected basedon the correction norm obtained by learning, and signals following thecorrection are output as output signals.

[0174] The optimizing device 1 is configured of a processing unit 11made up of a correcting unit 21 and a learning unit 22, to which aresupplied operating signals corresponding to user operations, besidesinput signals which are subject to processing.

[0175] The operating signals are supplied from an operating unit 2. Thatis, the operating unit 2 is configured of, for example, rotary or slideknobs, switches, pointing devices, etc., and operating signalscorresponding to user operations are supplied to the processing unit 11making up the optimizing device 1.

[0176] Digital input signals, for example, are supplied to thecorrecting unit 21 making up the optimizing device 1, as well as, forexample, correction parameters serving as correction norm for correctinginput signals, from the learning unit 22. The correcting unit 21corrects input signals based on the correction parameters (signalprocessing), and outputs signals following the correction, as outputsignals.

[0177] As well as operating signals from the operating unit 2 beingsupplied to the learning unit 22, input signals or output signals arealso supplied thereto, as necessary. The learning unit 22 monitors theoperating signals, and judges whether or not these can be used forlearning. In the event that the operating signals are learning operatingsignals which can be used for learning, the learning unit 22 learnscorrection parameters used for correcting input signals using the inputsignals or the output signals based on the learning operating signals,as necessary, and supplies these to the correcting unit 21.

[0178] Note that the learning unit 22 has built-in learning data memory53 and learning information memory 55, wherein the learning data memory53 stores learning data used for learning, and the learning informationmemory 55 stores later-described learning information obtained bylearning.

[0179] Next, the processing (optimizing processing) which the optimizingdevice 1 shown in FIG. 2 performs, will be described with reference tothe flowchart shown in FIG. 3.

[0180] First, in step S1, the learning unit 22 judges whether or notlearning operating signals have been received from operating unit 2.Here, in the event of operating the operating unit 2, usually, the userfirst performs general operations, and then performs detailed operationswhile confirming the output signals output according to the operations,and then stops operations at the point that output signals which theuser thinks are ultimately optimal are obtained. The operating signalscorresponding to the position of the operating unit 2 at the point whereoutput signals which the user thinks are optimal are obtained are thelearning operating signals, and accordingly, in the event that theoperating unit 2 is operated for a predetermined amount of time orlonger and then the operating is stopped, the learning unit 22 judgesthe operating signals at the time of stopping as being learningoperating signals.

[0181] In step S1, in the event that judgement is made that learningoperating signals are not received, i.e., in the event that the user hasnot operated the operating unit 2, or has operated but is still lookingfor an optimal position, for example, step S2 through step S10 areskipped, the flow proceeds to step S11, and the correcting unit 21corrects the input signals according to the correction parametersalready set, output signals are output as the correction results, andthe flow returns to step S1.

[0182] Also, in step S1, in the event that judgement is made thatlearning operating signals have been received, the flow proceeds to stepS2, the learning unit 22 obtains learning data used for learning, basedon the learning operating signals, and the flow proceeds to step S3. Instep S3, the learning data memory 53 stores the newest learning dataobtained in step S2.

[0183] Here, the learning data memory 53 has storage capacity capable ofstoring multiple sets of learning data. Also, in the event that thisstorage capacity is filled up, the learning data memory 53 is arrangedso as to overwrite the oldest stored values with the next learning data.Accordingly, there are multiple sets of learning data in the learningdata memory 53, with the newest stored.

[0184] Following storing of the learning data in the learning datamemory 53 in step S3, the flow proceeds to step S4, where the learningunit 22 performs learning using the newest learning data stored in thelearning data memory 53 and the learning information stored in thelearning information memory 55, obtains correction parameters, andproceeds to step S5. In step S5, the learning unit 22 updates the storedcontents of the learning information memory 55 with new learninginformation obtained part way through the learning in step S4, and theflow proceeds to step S6.

[0185] In step S6, the learning unit 22 obtains a later-describedproperness which represents the properness of the correction parametersobtained in step S4, the flow proceeds to step S7, and judgment is maderegarding whether or not the correction parameters obtained in step S4are proper, based on the properness.

[0186] In the event that judgement is made in step S7 that thecorrection parameters are proper, steps S8 and S9 are skipped, the flowproceeds to step S10, and a learning unit 22 outputs the correctionparameters judged to be proper to the correcting unit 21, and the flowproceeds to step S11. Accordingly, in this case, subsequent input signalcorrection is performed at the correcting unit 21 following thecorrection parameters newly obtained by the learning in step S4.

[0187] On the other hand, in the event that judgment is made in step S7that the correction parameters are not proper, the flow proceeds to stepS8, the learning unit 22 performs learning using only the newestlearning data of the learning data stored in the learning data memory53, obtains correction parameters, and proceeds to step S9. In step S9,the learning unit 22 updates the stored contents of the learninginformation memory 55 with the new learning information obtained partway through the learning in step S8, and the flow proceeds to step S10.In this case, in step S10, the learning unit 22 outputs the correctionparameters obtained only from the newest learning data in step S8 to thecorrecting unit 21, and the flow proceeds to step S11. Accordingly, inthis case, subsequent input signal correction is performed at thecorrecting unit 21 following the correction parameters newly obtained bythe learning in step S8.

[0188] Next, FIG. 4 illustrates a detailed configuration example of acase wherein the processing unit 11 shown in FIG. 2 is applied to an NRcircuit which removes noise from image signals and audio signals, forexample.

[0189] Weight memory 31 stores a weight (coefficient) W (e.g., a valuethat is 0 or greater but 1 or smaller) as correction parameters suppliedfrom a later-described selecting unit 41 of the learning unit 22. Theweight memory 32 stores weight 1−W supplied from the computing unit 33.

[0190] The computing unit 33 supplies a subtracted value 1−W wherein theweight W supplied from the selecting unit 41 of the learning unit 22 issubtracted from 1.0, which is supplied to the weighting memory 32 asweight. The computing unit 34 multiplies the input signals and theweight 1−W stored in the weight memory 32, and supplies the multipliedvalue to a computing unit 36. A computing unit 35 multiplies the weightW stored in the weight memory 31 and the output signals stored (latched)at the latch circuit 37, and supplies the multiplied value thereof tothe computing unit 36. The computing unit 36 adds the outputs of thecomputing units 34 and 35, and outputs the added value thereof as outputsignals.

[0191] A latch circuit 37 latches on the output signals which thecomputing unit 36 outputs, and supplies these to the computing unit 35.

[0192] With the embodiment shown in FIG. 4, the correcting unit 21 ofthe processing unit 11 is configured of the above weight memory 31 and32, computing units 33, 34, 35, and 36, and the latch circuit 37.

[0193] This selecting unit 41 selects one or the other of the weightoutput by the weight correcting unit 46 or the weight output by theoperating signal processing unit 50, and supplies this to the correctingunit 21 as correction parameters.

[0194] The input reliability calculating unit 42 is arranged so as to besupplied with input signals, and obtains the input reliability whichrepresents the reliability of the input signals thereof, which areoutput to the output reliability calculating unit 43 and the weightingcalculating unit 45. The output reliability calculating unit 43 obtainsoutput reliability representing the reliability of the output signals,based on the input reliability from the input reliability calculatingunit 42, and supplies this to the latch circuit 44 and the weightingcalculating unit 45. The latch circuit 44 stores (latches) the outputreliability from the output reliability computing unit 43, and suppliesthis to the output reliability calculating unit 43 and the weightingcalculating unit 45.

[0195] The weighting calculating unit 45 calculates the weighting fromthe input reliability from the input reliability calculating unit 42 andthe output reliability from the output reliability calculating unit 43,and outputs this to the weighting correcting unit 46. The weightingcorrecting unit 46 is arranged so as to be supplied from parametercontrol data memory 57 with parameter control data which controls theweighting serving as correction parameters, in addition to weight, andthe weighting correcting unit 46 processes (corrects) the weighting withthe parameter control data, and supplies this to the selecting unit 41.

[0196] An operating signal processing unit 50 is arranged so as to besupplied with operating signals from the operating unit 2 (FIG. 2), withthe operating signal processing unit 50 processing the operating signalssupplied thereto, and supplying the weight corresponding to theoperating signals thereof to the selecting unit 41, a tutor datagenerating unit 51, and a student data generating unit 52. Further, theoperating signal processing unit 50 judges whether or not the operatingsignals are the above-described learning operating signals, and in theevent that the operating signals are learning operating signals, a flagto that effect (hereafter may be referred to as learning flag) is addedto the weight to be output.

[0197] Upon receiving the weight with a learning flag added thereto fromthe operating signal processing unit 50, the tutor data generating unit51 generates tutor data which serves as a tutor for learning, andsupplies this to the learning data memory 53. That is, the tutor datagenerating unit 51 supplies the weight to which a learning flag has beenadded, to the learning data memory 53, as tutor data.

[0198] The student data generating unit 52, upon receiving the weightwith the learning flag added from the operating signal processing unit50, generates students data to serve as a student for learning, andsupplies this to the learning data memory 53. That is, the student datagenerating unit 52 is configured in the same way as the above-describedinput reliability calculating unit 42, output reliability calculatingunit 43, latch circuit 44, and weighting calculating unit 45, tocalculate weighting from input signals supplied thereto, and supplyingthe weighting calculated from the input signals upon reception ofweighting with a learning flag added thereto to the learning data memory53, as student data.

[0199] The learning data memory 53 stores a set of tutor data serving asweighting corresponding to learning operating signals, supplied from thetutor data generating unit 51, and student data serving as weightingcalculated from the input signals at the time of reception of thelearning operating signals, supplied from the student data generatingunit 52, as one set of learning data. As described above, the learningdata memory 53 is capable of storing multiple sets of learning data, andfurther, upon storing learning data to the storage capacity, is arrangedso as to store the next learning data by overwriting the oldest storedvalues. Accordingly, the learning data memory 53 basically has severalof the newest learning data sets stored therein at all times.

[0200] A parameter control data calculating unit 54 learns the tutordata and student data serving as learning data stored in the learningdata memory 53, as well as, if necessary, parameter control data whichminimizes a predetermined statistical error by computing new learninginformation using the learning information stored in the learninginformation memory 55, under control of a judgment control unit 56,which is supplied to the judgment control unit 56. Also, the parametercontrol data calculating unit 54 updates the stored contents of thelearning information memory 55 with new learning information obtained bylearning. The learning information memory 55 stores the learninginformation from the parameter control data calculating unit 54.

[0201] The judgment control unit 56 judges the properness of theparameter control data supplied from the parameter control datacalculating unit 54, by making reference to the newest learning datastored in the learning data memory 53. Also, the judgment control unit56 controls the parameter control data calculating unit 54 and suppliesthe parameter control data supplied from the parameter control datacalculating unit 54, to the parameter control data memory 57. Theparameter control data memory 57 updates the stored contents thereof bythe parameter control data supplied from the judgment control unit 56,and supplies this to the weighting correcting unit 46.

[0202] With the embodiment shown in FIG. 4, the learning unit 22 of theprocessing unit 11 is configured of the above-described selecting unit41 through weighting correcting unit 46, and the operating signalprocessing unit 50 through parameter control data memory 57.

[0203] With the processing unit 11 of the optimizing device 1 serving asan NR circuit configured as described above, noise in input signals isremoved as follows.

[0204] That is, for example, to simplify description, consideringremoving noise which fluctuates over time with regard to input signalshaving a constant true value upon which noise which fluctuates over timeis superimposed, as shown in FIG. 5A, by taking the average thereof,noise can be effectively removed by making the weighting small (notgiving much consideration) for input signals with a high noise level(i.e., signals with poor S/N), for example, as the degree of noise,while making the weighting great for input signals with a low noiselevel (i.e., signals with good S/N).

[0205] Accordingly, with the NR circuit shown in FIG. 4, noise can beeffectively removed by obtaining the closeness of the input signals tothe true value as an evaluation value of input signals, as shown in FIG.5B for example, i.e., the input reliability representing the reliabilitythat the input signals are the true value, and calculating the averagethereof by performing weighting corresponding to the input reliabilityfor the input signals.

[0206] Accordingly, with the NR circuit shown in FIG. 4, the weightaverage using the weighting corresponding to the input reliabilitythereof is obtained with regard to the input signals, and output asoutput signals, and the output signals y(t) can be obtained with thefollowing Expression, wherein the input signals, the output signals, andinput reliability, at time t, are represented by x(t), y(t), andα_(x(t)). $\begin{matrix}{\left\lbrack {{Expression}\quad 1} \right\rbrack {{y(t)} = \frac{\sum\limits_{i = 0}^{t}{\alpha_{x{(i)}}{x(i)}}}{\sum\limits_{i = 0}^{t}\alpha_{x{(i)}}}}} & (1)\end{matrix}$

[0207] Here, the greater the input reliability α_(x(t)) is, the greaterthe weighting provided is.

[0208] From Expression (1), the output signal y(t−1) from one sampleback from the current time t can be obtained from the followingExpression. $\begin{matrix}{\left\lbrack {{Expression}\quad 2} \right\rbrack {{y\left( {t - 1} \right)} = \frac{\sum\limits_{i = 0}^{t - 1}{\alpha_{x{(i)}}{x(i)}}}{\sum\limits_{i = 0}^{t - 1}\alpha_{x{(i)}}}}} & (2)\end{matrix}$

[0209] Also, with regard to the output signals y(t) as well, outputreliability aα_(y(t)) which represents the closeness to the true value,i.e., the reliability that the output signals y(t) are the true value,is introduced as the evaluation value of the output signals y(t), andthe output reliability α_(y(t−1)) for the output signal y(t−1) from onesample back from the current time t is defined by the followingExpression. $\begin{matrix}{\left\lbrack {{Expression}\quad 3} \right\rbrack {{\alpha_{y}\left( {t - 1} \right)} = {\sum\limits_{i = 0}^{t - 1}\alpha_{x{(i)}}}}} & (3)\end{matrix}$

[0210] In this case, from the Expressions (1) through (3), the outputsignals y(t) and the output reliability α_(y(t)) thereof can beexpressed as follows. $\begin{matrix}{\left\lbrack {{Expression}\quad 4} \right\rbrack \begin{matrix}{{y(t)} = \frac{\overset{\overset{\alpha_{y{({t - 1})}}{y{({t - 1})}}}{}}{\sum\limits_{i = 0}^{t - 1}{\alpha_{x{(i)}}{x(i)}}} + {\alpha_{x{(t)}}{x(t)}}}{\underset{\underset{\alpha_{y{({t - 1})}}}{}}{\sum\limits_{i = 0}^{t - 1}\alpha_{x{(i)}}} + \alpha_{x{(t)}}}} \\{= \frac{{\alpha_{y{({t - 1})}}{y\left( {t - 1} \right)}} + {\alpha_{x{(t)}}{x(t)}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}}\end{matrix}} & (4)\end{matrix}$

[0211] [Expression 5]

α_(y(t))=α_(y(t−1))+α_(x(t))  (5)

[0212] Also, the weight used for obtaining the output signal y(t) at thetime t is represented by w(t), and is defined by the followingExpression. $\begin{matrix}{\left\lbrack {{Expression}\quad 6} \right\rbrack {{w(t)} = \frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}}} & (6)\end{matrix}$

[0213] From Expression (6), the following Expression holds.$\begin{matrix}{\left\lbrack {{Expression}\quad 7} \right\rbrack {{1 - {w(t)}} = \frac{\alpha_{x{(t)}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}}} & (7)\end{matrix}$

[0214] Using Expressions (6) and (7), the output signals y(t) inExpression (4) can be represented by a weighted average from thefollowing multiplication and addition.

[0215] [Expression 8]

y(t)=w(t)y(t−1)+(1−w(t))x(t)  (8)

[0216] Note that the weight w(t) (and 1−w(t)) used in Expression (8) canbe obtained from Expression (6), from the output reliability α_(y(t−)1)for the output signal y(t−1) from one sample back, and the inputreliability α_(y(t)) for the current input signal x(t). Also, the outputreliability α_(y(t)) for the current output signal y(t) in Expression(5) can also be obtained from the output reliability α_(y(t−1)) for theoutput signal y(t−1) from one sample back, and the input reliabilityα_(x(t)) for the current input signal x(t).

[0217] Now, using the inverse numbers of the respective variancesα_(x(t)) ² or α_(y(t)) ² as the input reliability α_(x(t)) for the inputsignals x(t) or the output reliability α_(y(t)) for the output signalsy(t), i.e., defining the input reliability α_(x(t)) and the outputreliability α_(y(t)) as in the following Expression, $\begin{matrix}{\left\lbrack {{Expression}\quad 9} \right\rbrack \begin{matrix}{\alpha_{x{(t)}} = \frac{1}{\sigma_{x{(t)}}^{2}}} \\{\alpha_{y{(t)}} = \frac{1}{\sigma_{y{(t)}}^{2}}}\end{matrix}} & (9)\end{matrix}$

[0218] the weight w(t) in Expression (6) and the weight 1−w(t) inExpression (7) can be obtained from the following Expressions.$\begin{matrix}{\left\lbrack {{Expression}\quad 10} \right\rbrack {{w(t)} = \frac{\sigma_{x{(t)}}^{2}}{\sigma_{y{({t - 1})}}^{2} + \sigma_{x{(t)}}^{2}}}} & (10) \\{\left\lbrack {{Expression}\quad 11} \right\rbrack {{1 - {w(t)}} = \frac{\sigma_{y{({t - 1})}}^{2}}{\sigma_{y{({t - 1})}}^{2} + \sigma_{x{(t)}}^{2}}}} & (11)\end{matrix}$

[0219] Further, σ_(y(t)) ² can be obtained by the following Expression.

[0220] [Expression 12]

α_(y(t)) ² =w(t)² σ_(y(t−1)) ²+(1−w(t))² σ_(x(t)) ²  (12)

[0221] The NR circuit shown in FIG. 4 basically follows Expression (6)to perform correction parameter computing processing for computingcorrection parameters serving as the weight w(t), and uses the weightw(t) to calculate the weighting average between the output signal y(t−1)from one sample back and the current input signal x(t) followingExpression (8), thereby performing correcting processing for effectivelyremoving noise contained in the input signals x(t).

[0222] Now, the user will not necessarily feel that the output signalsobtained as the result of correction processing of the input signals bythe weight w(t) obtained following Expression (6) are optimal.Accordingly, the NR circuit shown in FIG. 4 performs control datalearning processing for obtaining parameter control data for controlling(correcting) the weight w(t) serving as the correction parameter, bylearning the operations of operating unit 2 by the user, and performscorrection processing of input signals using weighting corrected by theparameter control data.

[0223] The control data learning processing is performed as follows.

[0224] That is, a weight W_(i) corresponding to learning operatingsignals provided at the i′th time by the user operating the operatingunit 2 can be considered to be that which the user thinks to be optimalwith regard to the input signals input at the time that the learningoperating signals are provided, and accordingly, obtaining parametercontrol data which can correct the weight w(t) obtained followingExpression (6) to a value near the weight W_(i) corresponding to thelearning operating signals (ideally, the same value) is sufficient forthe control data learning processing.

[0225] To this end, let us now consider taking the weight w(t) obtainedfollowing Expression (6) as student data serving as a student forlearning, and the weight W_(i) corresponding to the learning operatingsignals as tutor data serving as a tutor for learning, and obtaining,from the weight w(t) serving as student data, a prediction value W_(i)′for the weight W_(i) serving as tutor data predicted by a linearexpression defined by a parameter control data a and b such as shown inthe following Expression.

[0226] [Expression 13]

W _(i) ′=aw _(i) +b  (13)

[0227] Note that in Expression (13) (as with the later-describedExpression (14) and Expressions (16) through (21)), w_(i) represents theweight w(t) serving as student data obtained following Expression (6) asto input signals input at the time that the weight W_(i) correspondingto the learning operating signals serving as tutor data is input.

[0228] From Expression (13), the error (prediction error) e_(i) betweenW_(i) serving as the tutor data and the prediction value W_(i)′ thereofis represented as shown in the following Expression. $\begin{matrix}{\left\lbrack {{Expression}\quad 14} \right\rbrack \begin{matrix}{e_{i} = {W_{i} - W_{i}^{\prime}}} \\{= {W_{i} - \left( {{aw}_{i} + b} \right)}}\end{matrix}} & (14)\end{matrix}$

[0229] Now, let us consider obtaining parameter control data a and bwhich minimizes the sum of the square (second power) error of theprediction error e_(i) in Expression (14) as represented in thefollowing Expression, $\begin{matrix}{\left\lbrack {{Expression}\quad 15} \right\rbrack {\sum\limits_{i = 1}^{n}e_{i}^{2}}} & (15)\end{matrix}$

[0230] using the least-square method (also called the least-square errormethod). Note that in Expression (15) (as with the later-describedExpressions (16) through (21)), N represents the number of sets of tutordata and student data.

[0231] First, partial differentiation of the sum of the square error inExpression (15) with each of the parameter control data a and b yieldsthe following expressions. $\begin{matrix}{\text{[Expression~~16]}{\frac{\partial{\sum\limits_{i = 1}^{N}\quad e_{i}^{2}}}{\partial a} = {{- 2}{\sum\limits_{i = 1}^{N}{w_{i}\left( {W_{i} - \left( {{a\quad w_{i}} + b} \right)} \right)}}}}} & \text{(16)} \\{\text{[Expression~~17]}{\frac{\partial{\sum\limits_{i = 1}^{N}\quad e_{i}^{2}}}{\partial b} = {{- 2}{\sum\limits_{i = 1}^{N}\left( {W_{i} - \left( {{a\quad w_{i}} + b} \right)} \right)}}}} & (17)\end{matrix}$

[0232] Since the minimum value (minimal value) of the sum of the squareerror in Expression (15) can be obtained by a and b which make theright-hand side in Expressions (16) and (17) to be 0, setting theright-hand side in both Expressions (16) and (17) to 0 yields Expression(18) from Expression (16), and Expression (19) from Expression (17),respectively. $\begin{matrix}{\text{[Expression~~18]}{{N{\sum\limits_{i = 1}^{N}{w_{i}W_{i}}}} = {{{Nb}{\sum\limits_{i = 1}^{N}w_{i}}} + {{Na}{\sum\limits_{i = 1}^{N}w_{i}^{2}}}}}} & (18) \\{\text{[Expression~~19]}{{Nb} = {{\sum\limits_{i = 1}^{N}w_{i}} - {a{\sum\limits_{i = 1}^{N}w_{i}}}}}} & (19)\end{matrix}$

[0233] Substituting Expression (19) into Expression (18) allows theparameter control data a to be obtained from the following Expression.$\begin{matrix}{\text{[Expression~~20]}{a = \frac{{N{\sum\limits_{i = 1}^{N}{w_{i}W_{i}}}} - {\sum\limits_{i = 1}^{N}{w_{i}{\sum\limits_{i = 1}^{N}W_{i}}}}}{{N{\sum\limits_{i = 1}^{N}w_{i}^{2}}} - \left( {\sum\limits_{i = 1}^{N}w_{i}} \right)^{2}}}} & (20)\end{matrix}$

[0234] Also, from Expressions (19) and (20), the parameter control datab can be obtained from the following Expression. $\begin{matrix}{\text{[Expression~~21]}{b = \frac{{\sum\limits_{i = 1}^{N}W_{i}} - {a{\sum\limits_{i = 1}^{N}w_{i}}}}{N}}} & (21)\end{matrix}$

[0235] With the NR circuit shown in FIG. 4, control data learningprocessing is performed for obtaining parameter control data a and b, asdescribed above.

[0236] Next, the correction processing, correction parameter computingprocessing, and control data learning processing, which the NR circuitshown in FIG. 4 performs, will be described with reference to theflowcharts in FIG. 6 through FIG. 8.

[0237] First, correction processing will be described with reference tothe flowchart in FIG. 6.

[0238] Upon weight w(t) serving as correction parameters being suppliedfrom the selecting unit 41 of the learning unit 22 to the correctingunit 21, the weight memory 31 of the correcting unit 21 stores theweight w(t) in the form of overwriting. Further, the computing unit 33of the correcting unit 21 subtracts the weight w(t) from 1.0, obtainsthe weight 1−w(t), supplies this to the weight memory 32, and storesthis in the form of overwriting.

[0239] Then, upon input signals x(t) being supplied, in step S21 thecomputing unit 34 computes the product of the input signals x(t) and theweight 1−w(t) stored in the weight memory 32, and supplies this to thecomputing unit 36. Further, in step S21, the computing unit 35 computesthe product of the weight w(t) stored in the weight memory 31 and outputsignal y(t−1) from one sample back latched by the latch circuit 37, andsupplies this to the computing unit 36.

[0240] The flow then proceeds to step S22, where the computing unit 36adds the product of the input signal x(t) and the weight 1−w(t), withthe product of the weight w(t) and the output signal y(t−1), therebyobtaining the weighted addition value (1−w(t)) x(t)+w(t) y(t−1) of theinput signal x(t) and output signal y(t−1), which is output as theoutput signal y(t). This output signal y(t) is also supplied to thelatch circuit 37, and the latch circuit 37 stores the output signal y(t)in the form of overwriting. Subsequently, the flow returns to step S21,waits for input signals of the next sample to be supplied, andsubsequently, the same processing is repeated.

[0241] Next, correction parameter computing processing will be describedwith reference to the flowchart shown in FIG. 7.

[0242] With correction parameter computing processing, first, in stepS31, the input reliability calculating unit 42 obtains input reliabilityα_((t)) based on dispersion of the input signals, for example.

[0243] That is, the input reliability calculating unit 42 has built-inunshown FIFO (first in first out) memory which is capable of latching,besides a sample x(t) of the current input signal, several past samples,with the dispersion thereof being calculated using the sample x(t) ofthe current input signal and several past samples, the inverse numberthereof being obtained as the input reliability α_(x(t)′) and suppliedto the output reliability calculating unit 43 and weight calculatingunit 45. Note that there are cases wherein there is not a sufficientnumber of input signal samples necessary for calculating dispersionimmediately following starting input of the input signals, but in suchcases, a default value, for example, is output as the input reliabilityα_(x(t)).

[0244] Subsequently, the flow proceeds to step S32, and the weightcalculating unit 45 obtains the weight w(t) following Expression (6),using the input reliability α_(x(t)) from the input reliabilitycalculating unit 42.

[0245] That is, at the timing that the input reliability α_(x(t)) issupplied from the input reliability calculating unit 42 to the weightcalculating unit 45, the output reliability α_(y(t−1)) which the outputreliability calculating unit 43 has output one sample back is latched atthe latch circuit 44, and in step S32, the weight calculating unit 45obtains the weight w(t) following Expression (6) using the inputreliability α_(x(t)) from the input reliability calculating unit 12 andthe output reliability α_(y(t−1)) latched by the latch circuit 14. Thisweight w(t) is supplied to the weight correcting unit 46.

[0246] Subsequently, the flow proceeds to step S33, and the weightcorrecting unit 46 reads the parameter control data out from theparameter control data memory 57, and proceeds to step S34. In step S34,the weight correcting unit 46 judges whether or not the parametercontrol data read out from the parameter control data memory 57 is dataof a mode wherein the weight w(t) is not corrected, i.e., auto mode datarepresenting a mode wherein the weight w(t) is obtained automatically,so to speak, from the input reliability and output reliability at theweight calculating unit 45 regardless of operations by the user at theoperating unit 2 is used without change as weight W for correcting theinput signals x(t) (hereafter may be referred to as auto mode).

[0247] In step S34, in the event that judgement is made that theparameter control data is not auto mode data, the flow proceeds to stepS35, the weight correcting unit 46 corrects the weight w(t) suppliedfrom the weight correcting unit 45 following the linear expression inExpression (13) defined by the parameter control data a and b suppliedfrom the parameter control data memory 57, and proceeds to step S36. Atstep S36, the weight correcting unit 46 supplies the post-correctionweight to the selecting unit 41, and proceeds to step S37. Here, inExpression (13), w_(i) is equivalent to the weight w(t) supplied fromthe weight calculating unit 45, and W_(i)′ is equivalent to thepost-correction weight W.

[0248] On the other hand, in step S34, in the event that judgement ismade that the parameter control data is auto mode data, the flow skipsstep S35, proceeds to step S36, where the weight correcting unit 46supplies the weight-w(t) from the weight calculating unit 45 to theselecting unit 41 without change, and the flow proceeds to step S37.

[0249] In step S37, the output reliability calculating unit 43 updatesthe output reliability. That is, the output reliability calculating unit43 adds, following Expression (5), the input reliability α_(x(t)) whichthe input reliability calculating unit 42 has calculated at theimmediately previous step S31, and the output reliability α_(y(t−1))from one sample back which the latch circuit 44 has latched, therebyobtaining the current output reliability α_(y(t)), which is stored inthe latch circuit 44 in the form of overwriting.

[0250] Then, the flow proceeds to step S38, and the selecting unit 41judges whether or not the operating unit 2 is being operated by theuser, from the output of the operating signal processing unit 50. In theevent that judgement is made that the operating unit 2 is not beingoperated in step S38, the flow proceeds to step S39, where the selectingunit 41 selects the weight supplied from the weight correcting unit 46(hereafter may be referred to as correcting weight), which is output tothe correcting unit 21, and the flow returns to step S31.

[0251] Also, in step S38, in the event that judgement is made that theoperating unit 2 is being operated, the flow proceeds to step S40, wherethe selecting unit 41 selects the weight to be output by the operatingsignal processing unit 50 according to the operation, outputs this tothe correcting unit 21, and the flow returns to step S31.

[0252] Accordingly, in the correction parameter computing processingshown in FIG. 7, in the event that the operating unit 2 is not beingoperated, the correcting weight is supplied to the correcting unit 21,while in the event that the operating unit 2 is being operated,weighting corresponding to the operating signals is supplied to thecorrecting unit 21. As a result, at the correcting unit 21, in the eventthat the operating unit 2 is not being operated, the input signals arecorrected by the correction weighting, and in the event that theoperating unit 2 is being operated, the input signals are corrected by aweighting corresponding to the operating signals.

[0253] Further, in the correction parameter computing processing in FIG.7, in the case of auto mode, weighting to be used for correctionprocessing is obtained only from the input reliability and outputreliability, regardless of the operations of operating unit 2, and ifnot in the auto mode, weighting to be used for correction processing isobtained using the parameter control data obtained by learning in thecontrol data learning processing shown in FIG. 8, which will bedescribed later, based on the operation of operating unit 2.

[0254] Next, the control data learning processing will be described withreference to the flowchart shown in FIG. 8.

[0255] In the control data learning processing, first, in step S41, theoperating signal processing unit 50 judges whether or not learningoperating signals have been received from the operating unit 2, and inthe event that judgement is made that these have not been received, theflow returns to step S41.

[0256] Also, in the event that judgement is made in step S41 thatlearning operating signals have been received from the operating unit 2,that is to say, in the event that judgment can be made that the user hasoperated the operating unit 2 so as to obtain a desired output signal,such as in cases wherein, for example, the operating unit 2 starts to beoperated, following which, without elapsing of a first time t1 orlonger, operating is performed continuously for a second time t2 orlonger, following which the operations thereof are stopped continuouslyfor a third time t3 or longer, or in cases wherein the operating unit 2starts to be operated, following which the operations thereof arestopped continuously for a third time t3 or linger, the flow proceeds tostep S42, where the tutor data generating unit 51 generates tutor data,while the student data generating unit 52 generates student data.

[0257] That is, in the event of receiving learning operating signals,the operating signal processing unit 50 supplies weight W correspondingto the learning operating signals (for example, weight W correspondingto the amount of operating of the operating unit 2 or the position ofknobs or levers serving as operating unit 2) to the tutor datagenerating unit 51 and student data generating unit 52, along with alearning flag. Upon receiving a weight W with a learning flag, the tutordata generating unit 51 obtains the weight W as tutor data, and suppliesthis to the learning data memory 53. Also the student data generatingunit 52, upon receiving the weight with a learning flag, obtains aweight w corresponding to the input signals at that time as studentdata, and supplies this to the learning data memory 53.

[0258] Here, the weight w corresponding to input signals means weightwhich is obtained automatically, so to speak, from the input reliabilityand output reliability, following Expression (6), and as describedabove, the student data generating unit 52 calculates this weight wcorresponding to the input signals, from the input signals.

[0259] Upon receiving the tutor data W from the tutor data generatingunit 51 and receiving the student data w from the student datagenerating unit 52, the learning data memory 53 stores the set of thenewest tutor data W and student data w in step S43, and proceeds to stepS44.

[0260] In step S44, the parameter control data calculating unit 54performs adding in for the least-square, with regard to the tutor dataand student data.

[0261] That is, the parameter control data calculating unit 54 performscomputation equivalent to the multiplication (w_(i)W_(i)) and summation(Σw_(i)W_(i)) of the student data w_(i) and the tutor data W_(i),computation equivalent to the summation (Σw_(i)) of the student dataw_(i), computation equivalent to the summation (ΣW_(i)) of the tutordata W_(i), and computation equivalent to the summation of the productof the student data w_(i) with itself (Σw_(i) ²), in Expressions (20)and (21).

[0262] Now, saying that, for example, N−1 sets of tutor data and studentdata have already been obtained, and that the N′th set of tutor data andstudent data has been obtained as the newest tutor data and studentdata, at this point adding in with regard to the N−1′th set of tutordata and student data has already been performed at the parametercontrol data calculating unit 54. Accordingly, with regard to the N′thset of tutor data and student data, as long as the results of the addingin which has already been performed with regard to the N−1′th set oftutor data and student data has been saved, the results of adding in theN sets of tutor data and student data can be obtained including thenewest tutor data and student data, by adding in the N′th set of tutordata and student data.

[0263] Accordingly, the parameter control data calculating unit 54 isarranged so as to store the previous adding in results in the learninginformation memory 55 as learning information, and so as to performadding in with regard to the N′th set of tutor data and student data,using this learning information. Now, the number of sets N of the tutordata and student data used for the adding in so far is also necessaryfor the adding in, and the learning information memory 55 is arranged soas to also store the number of sets N as learning information.

[0264] Following performing the adding-in in step S44, the parametercontrol data calculating unit 54 stores the results of adding in, in theform of overwriting in the learning information memory 55 as learninginformation, and the flow proceeds to step S45.

[0265] In step S45, the parameter control data calculating unit 54judges whether or not it is possible to obtain the parameter controldata a and b with the Expressions (20) and (21), from the results ofadding in serving as learning information stored in the learninginformation memory 55.

[0266] That is, with the set of tutor data and student data hereaftertaken as a learning pair where appropriate, the parameter control data aand b cannot be obtained from the Expressions (20) and (21) unlesslearning information obtained by at least two learning pairs exists.Accordingly, in step S45, judgement is made regarding whether or not itis possible to obtain the parameter control data a and b from thelearning information.

[0267] In the event that judgement is made in step S45 that obtainingthe parameter control data a and b is not possible, the parametercontrol data calculating unit 54 supplies notification to that effect tothe judgment control unit 56, and the flow proceeds to step S49. In stepS49, the judgment control unit 56 supplies auto mode data indicating theauto mode to the parameter control data memory 57, where it is stored asparameter control data. The flow then returns to step S41, andsubsequently, the same processing is repeated.

[0268] Accordingly, in the event that learning information sufficientfor obtaining the parameter control data a and b does not exist, theweight w(t) automatically obtained from the input reliability and outputreliability is used without change for correction of the input signalsx(t), as described with reference to FIG. 7.

[0269] On the other hand, in the event that judgement is made in stepS45 that the parameter control data a and b can be obtained, the flowproceeds to step S46, where the parameter control data calculating unit54 obtains the parameter control data a and b by calculating theExpressions (20) and (21) using the learning information, which issupplied to the judgment control unit 56, and the flow proceeds to stepS47.

[0270] In step S47, the judgment control unit 56 follows the linearexpression in Expression (13) defined by the parameter control data aand b from the parameter control data calculating unit 54 to obtain,from each piece of student data stored in the learning data memory 53,prediction values of corresponding tutor data, and the sum of the squareerror indicated in Expression (15) for the prediction error of theprediction value thereof (the error as to the tutor data stored in thelearning data memory 53), is obtained. Further, the judgment controlunit 56 obtains a normalization error wherein the sum of the squareerror is divided by the number of learning pairs stored in the learningdata memory 53, for example, and the flow proceeds to step S48.

[0271] In step S48, the judgment control unit 56 judges whether or notthe normalization error is greater than (equal to or more than) apredetermined threshold value S1. In step S48, in the event that thenormalization error is judged to be greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression inExpression (13) defined by the parameter control data a and b does notapproximate the relation between the student data and tutor data storedin the learning data memory 53 in a precise manner, the flow proceeds tostep S49, where the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored, as described above. The flowthen returns to step S41, and the same processing is repeated.

[0272] Accordingly, even if the parameter control data a and b can beobtained, in the event that the linear expression in Expression (13)defined by the parameter control data a and b does not approximate therelation between the student data and tutor data stored in the learningdata memory 53 in a precise manner, the weight w(t) automaticallyobtained from the input reliability and output reliability is usedwithout change for correction of the input signals x(t), in the samemanner as with cases wherein learning information sufficient forobtaining the parameter control data a and b does not exist.

[0273] On the other hand, in step S48, in the event that judgement ismade that the normalization error is not greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression inExpression (13) defined by the parameter control data a and bapproximates the relation between the student data and tutor data storedin the learning data memory 53 in a precise manner, the flow proceeds tostep S50, where the judgment control unit 56 obtains the error(distance) ε between the regression line represented by the linearexpression in Expression (13) defined by the parameter control data aand b from the parameter control data calculating unit 54, and the pointstipulated by the newest tutor data and student data stored in thelearning data memory 53.

[0274] The flow then proceeds to step S51, where the judgment controlunit 56 judges whether or not the magnitude of the error ε is greaterthan (equal to or more than) the predetermined threshold value S2, andin the event that judgement is made that it is not greater, step S52 isskipped, the flow proceeds to step S53, and the judgment control unit 56outputs the parameter control data a and b obtained in step S46 to theparameter control unit data memory 37. The parameter control data memory57 stores the parameter control data a and b from the judgment controlunit 56 in the form of overwriting, and the flow returns to step S41.

[0275] On the other hand, in the event that judgement is made in stepS51 that the magnitude of the error ε is greater than the predeterminedthreshold value S2, the flow proceeds to step S52, where the judgmentcontrol unit 56 recalculates the parameter control data a and b usingonly a predetermined number of learning pairs back from the newestlearning pair serving as the newest tutor data and student data storedin the learning data memory 53, by controlling the parameter controldata calculating unit 54 (without using the learning information in thelearning information memory 55). The flow then proceeds to step S53,where the judgment control unit 56 outputs the parameter control data aand b obtained in step S52 to the parameter control unit data memory 37,where it is stored in the form of overwriting, and the flow returns tostep S41.

[0276] Accordingly, in the event that the parameter control data a and bcan be obtained and also in the event that the linear expression inExpression (13) defined by the parameter control data a and bapproximates the relation between the student data and tutor data storedin the learning data memory 53 in a precise manner, the weight w(t)obtained from the input reliability and output reliability followingExpression (13) defined by the parameter control data a and b obtainedby learning being performed using the learning pairs obtained based onoperation of operating unit 2 by the user is corrected, and correctedweight W obtained by the correction thereof is used for correction ofthe input signals x(t).

[0277] Now, the regression line represented by the linear expression inExpression (13) defined by the parameter control data a and b obtainedin step S46 is a line which minimizes (the sum of) the square error withthe N points stipulated by the N sets of tutor data and student data asshown in FIG. 9A, and in step S50, the error ε between this line and thepoint stipulated by the newest tutor data and student data, is obtained.

[0278] In the event that the magnitude of this error ε is not greaterthan the threshold value S2, the regression line represented by thelinear expression in Expression (13) defined by the parameter controldata a and b obtained in step S46 can be thought to be approximatingeach of the points stipulated by tutor data and student data provided sofar, including the point stipulated with the newest tutor data andstudent data, in a relatively precise manner.

[0279] However, in the event that the magnitude of the error e isgreater than the threshold value S2, i.e., in the event that, as shownin FIG. 9B, the point stipulated by the newest tutor data and studentdata (indicated by a circle in FIG. 9B) greatly deviates from theregression line represented by the linear expression in Expression (13)defined by the parameter control data a and b obtained in step S46, itcan be conceived that the user has performed operations of the operatingunit 2 with a tendency that differs from those so far, for some reason.

[0280] Accordingly, in this case, the judging control unit 56recalculates the parameter control data a and b in step S52 using onlyseveral of the newest learning pairs of the learning pairs that arestored in the learning data memory 53, by controlling the parametercontrol data calculating unit 54.

[0281] That is to say, in this case, the parameter control datacalculating unit 54 recalculates the parameter control data a and bdefining the line in Expression (13) which most closely approximates thegroup of points stipulated by a tutor data and student data, using onlyseveral recent tutor data and student data sets, without using(forgetting) the learning information serving as past results of addingin that are stored in the learning information memory 55.

[0282] Specifically, as shown in FIG. 9C, for example, the parametercontrol data calculating unit 54 obtains parameter control data a′ andb′ defining a line which passes through a point stipulated by the newesttutor data and student data (indicated by a circle in FIG. 9C), and apoint stipulated by the tutor data and student data provided one timeback (indicated by triangle in FIG. 9C).

[0283] As described above, judgment is made regarding whether or notoperating signals supplied according to user operations can be used forlearning or not, and in the event that the signals are learningoperating signals which can be used for learning, parameter control dataa and b for correcting the weight for correcting input signals islearned based on the learning operating signals, so the operations ofthe user can be learned without the user knowing, and as a result,processing which is suitable for the user gradually comes to beperformed based on the learned results, ultimately performing processingof which is optimal for the user.

[0284] This means that, from the perspective of the user, after normallyoperating the operating unit 2 for a while, noise reduction resultswhich are optimal for the user can be obtained with regard to varioustypes of input signals even without making operations, and accordingly,the device becomes broken in, so to speak. At this stage of breaking inthe device, the user operates the operating unit 2 so as to obtaindesired output signals, so the relation between the operations of theoperating unit 2 and the weight W used for correcting input signalsgradually comes to be clearly recognized by the user, so ultimately, theoperations of the operating unit 2 by the user and the weight W used forcorrecting input signals come to be qualitatively correlated.

[0285] Also, at the NR circuit shown in FIG. 4, the weight W used forthe correction processing (FIG. 6) performed at the correcting unit 21following the operations of operating unit 2 by the user is changed sothat the desired output signals by the user can be obtained. That is,upon the user operating the operating unit 2, the operating signalprocessing unit 50 outputs a weight representing the operating signalscorresponding to the operations, the selecting unit 41 selects theweight, and supplies this to the correcting unit 21. In this case, atthe correcting unit 21, correction processing indicated by Expression(8) is performed using the weight corresponding to the operations madeby the user. In the event that the weight w(t) in Expression (8) is tobe changed, due to operations made by the user, the contents of theprocessing (correction processing) represented by Expression (8) arealso changed, as a matter of course, so it can be said he that at the NRcircuit shown in FIG. 4, the “contents of processing” are being changedfollowing the operations of the user so that output signals desired bythe user can be obtained.

[0286] Further, at the NR circuit shown in FIG. 4, in the event that theparameter control data a and b cannot be obtained, or in the event thatthis can be obtained but the linear expression in Expression (13)defined by the parameter control data a and b does not approximate therelation between the student data and tutor data stored in the learningdata memory 53 in a precise manner, weighting automatically obtainedfrom the input reliability and output reliability are used forcorrection processing at the correcting unit 21. On the other hand, inthe event that the parameter control data a and b can be obtained, andalso, the linear expression in Expression (13) defined by the parametercontrol data a and b approximates the relation between the student dataand tutor data stored in the learning data memory 53 in a precisemanner, the weighting obtained from the input reliability and outputreliability is corrected following Expression (13) defined by theparameter control data a and b obtained by learning being performedusing learning pairs obtained based on operations of operating unit 2 bythe user, and corrected weighting obtained by this correcting is usedfor correction processing by the correcting unit 21.

[0287] That is to say, with the NR circuit shown in FIG. 4, in the eventthat sufficient learning pairs have not been input from the user, or inthe event that learning pairs whereby highly precise approximation ispossible have not been input, weight automatically obtained from theinput reliability and output reliability is used for the correctionprocessing at the correcting unit 21, and in the event that learningpairs whereby highly precise approximation is possible have been input,correction weighting obtained by the parameter control data a and bobtained by learning being performed using the learning pairs is usedfor correction processing at the correcting unit 21.

[0288] Accordingly, the weight w(t) in Expression (8) does, after all,change between cases wherein a sufficient number of learning pairs orlearning pairs enabling highly precise approximation are not obtained,and cases wherein learning pairs enabling highly precise approximationare obtained, and consequently, the contents of correction processingrepresented by the Expression (8) also change, so from this perspectiveas well with the NR circuit shown in FIG. 4, it can be the that the“contents of processing” are changed so that output signals desired bythe user are obtained.

[0289] Further, with the NR circuit in FIG. 4, the system forcalculating weight used for the correction processing changes dependingon the cases wherein a sufficient number of learning pairs or learningpairs enabling highly precise approximation are not obtained, and caseswherein learning pairs enabling highly precise approximation areobtained.

[0290] That is to say, in cases wherein a sufficient number of learningpairs or learning pairs enabling highly precise approximation are notobtained, the weight is obtained from the input reliability and outputreliability, regardless of operations made by the user. On the otherhand, in the event that learning pairs enabling highly preciseapproximation has been obtained, the weighting is obtained based on theparameter control data obtained by learning using the learning pairsobtained based on the operations of the user.

[0291] Accordingly, in this case, it can be said that the processingsystem for calculating weighting, i.e., the algorithm for how to obtainweighting, is changed such that output signals desired by the user canbe obtained, following operations of the user.

[0292] Now, representing the processing for obtaining weight with afunction F, the above-described change of “contents of processing” isequivalent to the function F being changed. Here, cases of the functionF being changed are generally divided into cases wherein the form of thefunction F itself changes (e.g., cases wherein F=x changes to F=x²,etc.), and cases wherein the form of the function F itself does notchange but the coefficients defining the function F change (e.g., F=2xchanges to F=3x, etc.).

[0293] Now, of the changes in “contents of processing”, if we say thatcases wherein the form of the function F representing the processing ofitself changes are changes of the “structure of processing”, changingthe processing system for calculating weight, i.e., the algorithm forhow to obtain weight, as described above, can be said to be changing the“structure of processing”.

[0294] Accordingly, with the NR circuit shown in FIG. 4, the “contentsof processing”, and further, the “structure of processing” is alsochanged following operations made by the user, whereby output signalsdesired by the user can be obtained.

[0295] Also, as for input signals, image signals and audio signals maybe used, of course, and other signals as well. However, in the case thatthe input signals are image signals, the input reliability is calculatedbased on dispersion obtained from multiple pixels near a pixel to beprocessed, either space-wise or time-wise.

[0296] Also, with the above-described case, the weight w obtained frominput reliability and the like is corrected to the correcting weight Wby the linear expression in Expression (13) defined by the parametercontrol data a and b in the learning unit 22, in order to facilitatedescription, but actually, correction of the weight w is preferablyperformed by a higher expression. Also, the degree in the higherexpression is preferably set to a suitable value based on applicationsand the like to which the optimizing device 1 is applied, for example.

[0297] Further, an arrangement may be made for the expression forobtaining the correction weight W from the weight w (hereafter may bereferred to as correction weight calculating expression) whereinmultiple expressions are prepared besides the linear expression W=aw+bin Expression (13), such as, for example, the quadratic expressionW=aw²+bw+c, the cubic expression W=aw³+bw²+cw+d, and so forth (whereina, b, c, and d, are predetermined coefficients), and wherein that of themultiple correction weight calculating expressions which shields thesmallest normalization error is employed. Note that in this case, thecorrection weight calculating expression wherein the normalization errorobtained from the learning pair obtained by operations of the user issmallest, is selected, and the correction weighting is to be obtainedusing the selected correcting weight calculating expression. That is tosay, the algorithm for how to obtain the correction weight is changedfollowing the operations of the user. Accordingly, in this case as well,it can be said that the “structure of processing” is being changedfollowing operations of the user.

[0298] Next, FIG. 10 illustrates another detailed configuration exampleof a case wherein the processing unit 11 of the optimizing device 1shown in FIG. 4 is applied to an NR circuit. In the figure, the partscorresponding to the case in FIG. 4 are denoted with the same referencenumerals, and further description thereof will be omitted as suitable.That is to say, the NR circuit shown in FIG. 10 is basically the sameconfiguration as that shown in FIG. 4, other than not being providedwith the weight correcting unit 46, and an input reliability calculatingunit 61 and student data generating unit 62 being provided instead ofthe input reliability calculating unit 42 and student data generatingunit 52, respectively.

[0299] The input reliability calculating unit 61 calculates the inputreliability of the input signals from multiple samples of input signalsand parameter control data stored in the parameter control data memory57, and supplies this to the output reliability calculating unit 43 andweight calculating unit 45.

[0300] The student data generating unit 62 obtains the input signals,and the output reliability output from the output reliabilitycalculating unit 43, as student data, and supplies this to the learningdata memory 53.

[0301] Note that with the embodiment shown in FIG. 10, there is noweight correcting unit 46 provided, so the weight obtained in the weightcalculating unit 45 is applied to the selecting unit 41 with no change,and this selecting unit 41 is arranged so as to select and output one orthe other of the weight output by this weight calculating unit 45 andthe weight output by the operating signal processing unit 50, in thesame way as with that shown in FIG. 4.

[0302] Also, with the embodiment shown in FIG. 10, the parameter controldata functions as data to control input reliability.

[0303] As with the NR circuit shown in FIG. 4, the NR circuit in FIG. 10also performs correction processing, correction parameter computingprocessing, and control data learning processing. The correctionprocessing is performed in the same way as the processing described withreference to FIG. 6, so description of the correction processing will beomitted for the NR circuit shown in FIG. 10, and description will bemade regarding the correction parameter computing processing and controldata learning processing.

[0304] That is to say, with the NR circuit in FIG. 10, correctionparameter computing processing and control data learning processing isperformed with the input reliability α_(x(t)) stipulating the weightshown in Expression (6) which is used in correction processing as beingdefined by the following expression, for example.

[0305] [Expression 22]

α_(x(t)) =a ₁ x ₁ +a ₂ x ₂ + . . . +a _(N) x _(N)  (22)

[0306] Note that, however, in Expression (22), a₁, a₂, . . . , a_(N)represent parameter control data, and x₁, x₂, . . . , x_(N) representsamples of input signals in a predetermined relation with samples ofinput signals which are to be subjected to processing (samples ofinterest). Now, in the event that the input signals are image signalsfor example, a pixel serving as a sample of interest (indicated by an Xin FIG. 11) and a pixel at a position near that pixel space-wise ortime-wise (indicated by a circle in FIG. 11), can be used for x₁, x₂, .. . , x_(N), for example.

[0307] From Expression (22), the weight w(t) provided in Expression (6)can be represented as in Expression (23). $\begin{matrix}{{\text{[Expression~~23]}{{w(t)} = {\frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + \alpha_{x{(t)}}}\quad = \frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + \left( {{\alpha_{1}x_{1}} + {a_{2}x_{2}} + \cdots + {a_{N}x_{N}}} \right)}}}}} & (23)\end{matrix}$

[0308] Accordingly, in the event that the input signals x₁, x₂, . . . ,x_(N), are input, parameter control data a₁, a₂, . . . , a_(N) whichsatisfies the following Expression should be obtained from Expression(23) in order to obtain the weight W provided from the user.$\begin{matrix}{\text{[Expression~~24]}{W = \frac{\alpha_{y{({t - 1})}}}{\alpha_{y{({t - 1})}} + \left( {{\alpha_{1}x_{1}} + {a_{2}x_{2}} + \cdots + {a_{N}x_{N}}} \right)}}} & (24)\end{matrix}$

[0309] As a variant of the Expression (24), Expression (25) can beobtained.

[0310] [Expression 25]

(a ₁ x ₁ +a ₂ x ₂ + . . . +a _(N) x _(N))W+(W−1)α_(y(t−1))=0  (25)

[0311] Obtaining the parameter control data a₁, a₂, . . . , a_(N) whichsatisfies Expression (25) at all times is generally difficult, so here,we will consider obtaining parameter control data a₁, a₂, . . . , a_(N)wherein, for example, the sum of the square error of the left-hand sideand right-hand side of the Expression (25) is smallest, by theleast-square method.

[0312] Here, making the sum of the square error of the left-hand sideand right-hand side of the Expression (25) to be smallest means makingthe square error of the weight w(t) provided by the Expression (23) andthe weight W provided by the user to be smallest, i.e., with the weightW provided by the user as tutor data, and the input signals x₁, x₂, . .. , x_(N), and output reliability α_(y(t−1)) defining the weight w(t) inExpression (23) as student data, equivalent to making the square errorof the weight w(t) calculated from the student data by Expression (23)and the weight W serving as tutor data provided by the user to besmallest, and the weight w(t) calculated from such parameter controldata a₁, a₂, . . . , a_(N) and the student data by Expression (23) haslittle error with the tutor data W.

[0313] The square-error e² of the left-hand side and right-hand side ofExpression (25) is provided by Expression (26).

[0314] [Expression 26]

e ²={(a ₁ x ₁ +a ₂ x ₂ + . . . +a _(N) x _(N))W+(W−1)α_(y(t−1)}) ²  (26)

[0315] The parameter control data a₁, a₂, . . . , a_(N) which makes thesquare-error e² smallest is provided by the conditions that partialdifferentiation of the square-error e² in Expression (26) by each of theparameter control data a₁, a₂, . . . , a_(N) yields 0, i.e., by thefollowing Expression. $\begin{matrix}{\text{[Expression~~27]}{{\frac{\partial ^{2}}{\partial a_{1}} = 0},{\frac{\partial ^{2}}{\partial a_{2}} = 0},\cdots \quad,{\frac{\partial ^{2}}{a_{N}} = 0}}} & (27)\end{matrix}$

[0316] Substituting Expression (26) into Expression (27) yieldsExpression (28). $\begin{matrix}{\text{[Expression~~28]}{{\left( {{a_{1}x_{1}W} + {a_{2}x_{2}W} + \cdots + {a_{N}x_{N}W} + {\left( {W - 1} \right)\alpha_{y{({t - 1})}}}} \right){Wx}_{1}} = 0}{{\left( {{a_{1}x_{1}W} + {a_{2}x_{2}W} + \cdots + {a_{N}x_{N}W} + {\left( {W - 1} \right)\alpha_{y{({t - 1})}}}} \right){Wx}_{2}} = 0}\quad {{{\vdots \left( {{a_{1}x_{1}W} + {a_{2}x_{2}W} + \cdots + {a_{N}x_{N}W} + {\left( {W - 1} \right)\alpha_{y{({t - 1})}}}} \right)}{Wx}_{N}} = 0}} & (28)\end{matrix}$

[0317] Accordingly, defining the matrixes X, A, and Y each as shown inExpression (29), the relation shown in Expression (30) holds for thematrixes X, A, and Y, from the Expression (28). $\begin{matrix}{\text{[Expression~~29]}{{X = \begin{bmatrix}{{\sum{W^{2}x_{1}^{2}}}\quad} & {\sum{W^{2}x_{2}x_{1}}} & \cdots & {\sum{W^{2}x_{N}x_{1}}} \\{\sum{W^{2}x_{1}x_{2}}} & {{\sum{W^{2}x_{2}^{2}}}\quad} & \cdots & {\sum{W^{2}x_{N}x_{2}}} \\\vdots & \vdots & ⋰ & \vdots \\{\sum{W^{2}x_{1}x_{N}}} & {\sum{W^{2}x_{2}x_{N}}} & \quad & {\sum{W^{2}x_{N}^{2}}}\end{bmatrix}},{A = \begin{bmatrix}a_{1} \\a_{2} \\\vdots \\a_{N}\end{bmatrix}},{Y = \begin{bmatrix}{\sum{{W\left( {1 - W} \right)}\alpha_{y{({t - 1})}}x_{1}}} \\{\sum{{W\left( {1 - W} \right)}\alpha_{y{({t - 1})}}x_{2}}} \\\vdots \\{\sum{{W\left( {1 - W} \right)}\alpha_{y{({t - 1})}}x_{N}}}\end{bmatrix}}}} & (29)\end{matrix}$

[0318] [Expression 30]

XA=Y  (30)

[0319] Note that the summation (Σ) in Expression (29) means thesummation with regard to the number of sets of input signals x₁ throughx_(N) and the weight W provided from the user upon the input signals x₁through x_(N) being input from the user.

[0320] Expression (30) can be solved with regard to the matrix (vector)A, i.e., the parameter control data a₁, a₂, . . . , a_(N), by theCholesky method or the like, for example.

[0321] The NR circuit in FIG. 10 performs control data learningprocessing as described above, wherein, with the weight W provided bythe user as tutor data, and with the input signals x₁, x₂, . . . ,x_(N), and output reliability α_(y(t−1)) defining the weight w(t) inExpression (23) as student data, parameter control data a₁, a₂, . . . ,a_(N), minimizing the square error of the weight w(t) calculated fromthe student data by Expression (23) and the weight W serving as tutordata provided by the user is learned by the least-square. Further, theNR circuit in FIG. 10 obtains the input reliability α_(x(t)) from theExpression (22) defined by the parameter control data a₁ through a_(N),and further performs correction parameter computing processing forobtaining the weight serving as correction parameters, followingExpression (23), from the input reliability α_(x(t)) and outputreliability α_(y(t−1)).

[0322] Now, the correction parameter computing processing with the NRcircuit shown in FIG. 10 will be described with reference to theflowchart in FIG. 12.

[0323] In the correction parameter computing processing, first, in stepS61, the input reliability calculating unit 61 reads out the parametercontrol data from the parameter control data memory 57, and the flowproceeds to step S62. In step S62, the input reliability calculatingunit 61 judges whether or not the parameter control data read out fromthe parameter control data memory 57 is auto mode data which representsa mode wherein input reliability is obtained without using parametercontrol data, i.e., a mode wherein input reliability is obtainedautomatically, so to speak, from input signals alone (this mode also mayhereafter be referred to as auto mode), regardless of operations ofoperating unit 2 by the user.

[0324] In step S62, in the event that judgment is made that theparameter control data is not auto mode data, the flow proceeds to stepS63, the input reliability calculating unit 61 obtains using the newestN pieces of input signal samples x₁ through x_(N) supplied thereto,following the linear expression in the Expression (22) defined by theparameter control data a₁ through a_(N) read out from the parametercontrol data memory 57, supplies this to the output reliabilitycalculating unit 43 and weight calculating unit 45, and proceeds to stepS65.

[0325] Also, in the event that the parameter control data is judged tobe auto mode data in step S62, the flow proceeds to step S64, and theinput reliability calculating unit 61 obtains the input reliabilityα_(x(t)) using the input signals alone, based on dispersion thereof, inthe same manner as with step S31 in FIG. 7, for example, and suppliesthis to the output reliability calculating unit 43 and weightcalculating unit 45.

[0326] In step S65, the weight calculating unit 45 obtains the weightw(t) following Expression (23), using the input reliability α_(x(t))from the input reliability calculating unit 61, and the outputreliability α_(y(t−1)) output one sample back by the output reliabilitycalculating unit 43, which is latched at the latch circuit 44. Thisweight w(t) is supplied from the weight calculating unit 45 to theselecting unit 41.

[0327] Subsequently, the flow proceeds to step S66, and in the samemanner as with step S37 in FIG. 7, the output reliability calculatingunit 43 adds the input reliability α_(x(t)) supplied from the inputreliability calculating unit 61 and the output reliability α_(y(t−1))from one sample back, which is latched at the latch circuit 44 accordingto Expression (5), thereby updating the output reliability α_(y(t)), andstoring in the form of overwriting the latch circuit 44.

[0328] The flow then proceeds to step S67, where the selecting unit 41judges whether or not the operating unit 2 is being operated by theuser, from the output of operating signal processing unit 50. In theevent that judgment is made in step S67 that operating unit 2 is notbeing operated, the flow proceeds to step S68, the selecting unit 41selects the weight supplied from the weight calculating unit 45, outputsthis to the correcting unit 21, and the flow returns to step S61.

[0329] Also, in the event that judgment is made in step S67 that theoperating unit 2 is being operated, the flow proceeds to step S69, wherethe selecting unit 41 selects the weight output by the output signalprocessing unit 50 according to operations thereof, outputs this to thecorrecting unit 21, and the flow returns to step S61.

[0330] Accordingly, in the correction parameter computing processing inFIG. 12, in the event that the operating unit 2 is not being operated,weight calculated based on input reliability is supplied to thecorrecting unit 21, and in the event of operating unit 2 is beingoperated, weight corresponding to the operating signals thereof aresupplied to the correcting unit 21. Consequently, at the correcting unit21, in the event that the operating unit 2 is not being operated, inputsignals are corrected by weighting based on input reliability, and inthe event that operating unit 2 is being operated, input signals arecorrected by weighting corresponding to the operating signals thereof.

[0331] Further, with the correction parameter computing processing inFIG. 12, in the case of the auto mode, the weight to be used forcorrection processing is obtained from the input reliability based ondispersion of input signals regardless of operations of the operatingunit 2, and in the event of not being in the auto mode, the weight usedfor correction processing is obtained from input reliability obtainedusing the parameter control data obtained by learning with the controldata learning processing shown in FIG. 13 which shall be describedlater, based on the operations of operating unit 2.

[0332] Next, the control data learning processing which the NR circuitshown in FIG. 10 performs will be described with reference to theflowchart in FIG. 13.

[0333] With the control data learning processing, first, in step S71, inthe same manner as with step S41 in FIG. 8, the operating signalprocessing unit 50 judges whether or not learning operating signals havebeen received from the operating unit 2, and in the event that judgmentis made that these have not been received, the flow returns to step S71.

[0334] Also, in the event that judgment is made in step S71 thatlearning operating signals have been received from the operating unit 2,i.e., in the event that judgment can be made that the user has operatedthe operating unit 2 so as to obtain a desired output signal, such as incases wherein, for example, the operating unit 2 starts to be operated,following which, without elapsing of a first time t1 or longer,operating is performed continuously for a second time t2 or longer,following which the operations thereof are stopped continuously for athird time t3 or longer, or cases wherein following starting theoperations of the operating unit 2, operations thereof are stoppedcontinuously for a third time t3 or longer, the flow proceeds to stepS72, where the tutor data generating unit 51 generates tutor data, whilethe student data generating unit 62 generates student data.

[0335] That is, in the event of receiving learning operating signals,the operating signal processing unit 50 supplies weight W correspondingto the learning operating signals to the tutor data generating unit 51and student data generating unit 62, along with a learning flag. Uponreceiving a weight W with a learning flag, the data generating unit 51obtains the weight W as tutor data, and supplies this to the learningdata memory 53.

[0336] On the other hand, the student data generating unit 62 has abuilt-in buffer (not shown) for buffering input signals, and constantlystores input signals in the buffer to the storage capacity thereof, andin the event of receiving a weight with the learning flag, samples x₁through x_(N) of input signals in a predetermined positional relationwith samples of the input signals input at that time are read out fromthe built-in buffer. Further, the student data generating unit 62 readsout the output reliability α_(y(t−1)) from the output reliabilitycalculating unit 43. The student data generating unit 62 then suppliesthe samples x₁ through x_(N) of input signals and output reliabilityα_(y(t−1)) to the learning data memory 53 as student data.

[0337] Upon receiving the tutor data W from the tutor data generatingunit 51 and the student data x₁ through x_(N) and α_(y(t−1)) from thestudent data generating unit 62, in step S73 the learning data memory 53stores the set (learning pair) of the newest tutor data W, and studentdata x₁ through x_(N) and α_(y(t−1)) and the flow proceeds to step S74.

[0338] At step S74, the parameter control data calculating unit 54performs adding-in for least-square, for the tutor data and studentdata.

[0339] That is, the parameter control data calculating unit 54 performscomputation of the product between one student data with another whichare components of the matrix X and Y in Expression (29), the product ofthe student data and tutor data, and the summation thereof.

[0340] Now, the adding in step S74 is performed in the same manner asthat in step S44 in FIG. 8. That is to say, the results of the previousadding-in are stored in the learning information memory 55 as learninginformation, and the parameter control data calculating unit 54 usesthis learning information to perform adding in with regard to the newesttutor data and student data.

[0341] Following the adding in step S74, the parameter control datacalculating unit 54 stores the results of the adding-in within thelearning information memory 55, as learning information, in the form ofoverwriting, the flow proceeds to step S75, and the parameter controldata computing unit 54 judges whether or not Expression (30) can besolved with regard to matrix A from the adding results as learninginformation stored in the learning information memory 55, i.e., whetheror not the parameter control data a₁ through a_(N) can be obtained.

[0342] That is to say, Expression (30) cannot be solved with regard tomatrix A unless learning information obtained from a predeterminednumber of learning pairs or more exists, and the parameter control dataa₁ through a_(N) serving as the components thereof cannot be obtained.Accordingly, in step S75, whether or not the parameter control data a₁through a_(N) can be obtained from the learning information is judged.

[0343] In step S75, in the event that judgment is made that theparameter control data a₁ through a_(N) cannot be obtained, theparameter control data calculating unit 54 supplies information to thateffect to the judgment control unit 56, and the flow proceeds to stepS79. In step S79, the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored. The flow then returns tostep S71, and subsequently, the same processing is repeated.

[0344] Accordingly, in the event that sufficient learning informationfor obtaining the parameter control data a₁ through a_(N) does notexist, the weight obtained from the input reliability based ondispersion of input signals is used for correction of the input signalsx(t), as described with reference to FIG. 12.

[0345] On the other hand, in the event that judgment is made in step S75that the parameter control data can be obtained, the flow proceeds tostep S76, where the parameter control data calculating unit 54 obtainsthe parameter control data a₁ through a_(N) by solving Expression (30)with regard to matrix A using learning information, supplies this to thejudgment control unit 56, and the flow proceeds to step S77.

[0346] At step S77, the judgment control unit 56 follows the Expression(23) defined by the parameter control data a₁ through a_(N) from theparameter control data calculating unit 54 to obtain a prediction valueof corresponding tutor data from the student data stored in the learningdata memory 53, and obtains the sum of the square-error represented byExpression (26), of the prediction error (the error as to the tutor datastored in the learning data memory 53) of that prediction value.Further, the judgment control unit 56 obtains a normalization errorwherein the square-error sum is divided by the number of learning pairsstored in the learning data memory 53, for example, and the flowproceeds to step S78.

[0347] In step S78, the judgment control unit 56 judges whether or notthe normalization error is greater than (equal to or more than) apredetermined threshold value S1. In the event that judgment is made instep S78 that the normalization error is greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression in theExpression (23) defined by the parameter control data a₁ through a_(N)does not precisely approximate the relation between the student data andthe tutor data stored in the learning data memory 53, the flow proceedsto step S79, and the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored in the form of overwriting.The flow then returns to step S71, and subsequently, the same processingis repeated.

[0348] Accordingly, even in the event that the parameter control data a₁through a_(N) can be obtained, in the event that the Expression (23)defined by the parameter control data a₁ through a_(N) does notprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the weight obtained byinput reliability based on dispersion of input signals is used forcorrecting the input signals x(t), in the same way as cases whereinsufficient learning information for obtaining the parameter control dataa₁ through a_(N) does not exist.

[0349] On the other hand, in the event that judgment is made in step S78that the normalization error is not greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression in theExpression (23) defined by the parameter control data a₁ through a_(N)does precisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the flow proceeds tostep S80, and the judgment control unit 56 obtains the error (distance)ε between the plane (line) of the Expression (23) defined by theparameter control data a₁ through a_(N) obtained at the parametercontrol data calculating unit 54 and the point stipulated by the newesttutor data and student data stored in the learning data memory 53.

[0350] The flow then proceeds to step S81, where the judgment controlunit 56 judges whether or not the magnitude of the error ε is greaterthan (equal to or more than) a predetermined threshold value S2, and inthe event that judgment is made that this is not greater, step S82 isskipped, the flow proceeds to step S83, and the judgment control unit 56outputs the parameter control data a₁ through a_(N) obtained in step S76to the parameter control unit data memory 37. The parameter control datamemory 57 stores the parameter control data a₁ through a_(N) from thejudgment control unit 56 in the form of overwriting, the flow returns tostep S71, and subsequently, the same processing is repeated.

[0351] On the other hand, in the event that judgment is made in step S81that the magnitude of the error ε is greater than the predeterminedthreshold value S2, the flow proceeds to step S82, where the judgmentcontrol unit 56 controls the parameter control data calculating unit 54to re-calculate the parameter control data a₁ through a_(N) using onlythe newest tutor data and student data stored in the learning datamemory 53. The flow then proceeds to step S83, where the judgmentcontrol unit 56 outputs the parameter control data a₁ through a_(N)obtained in step S82 to the parameter control data memory 57, where itis stored in the form of overwriting, and the flow returns to step S71.

[0352] That is, in the embodiment shown in FIG. 13 as well, the error εbetween the plane defined by Expression (23) from the parameter controldata a₁ through a_(N) obtained from the tutor data and student dataprovided so far and the point stipulated by the newest tutor data andstudent data is obtained in step S82, in the same manner as with theembodiment in FIG. 8.

[0353] In the event that the magnitude of the error ε is not greaterthan the predetermined threshold value S2, the plane defined byExpression (23) from the parameter control data a₁ through a_(N)obtained in step S76 is considered to relatively precisely approximateall of the points stipulated by tutor data and student data provided sofar, including the point stipulated by the newest tutor data and studentdata, so this parameter control data a₁ through a_(N) is stored in theparameter control data memory 57.

[0354] On the other hand, in the event that the magnitude of the error εis greater than the predetermined threshold value S2, the plane ofExpression (23) defined by the parameter control data a₁ through a_(N)obtained in step S76 is considered to relatively greatly deviate fromthe points stipulated by the newest tutor data and student data, so thejudgment control unit 56 controls the parameter control data calculatingunit 54 so as to re-calculate the parameter control data a₁ througha_(N) in step S82 using only the newest tutor data and student datastored in the learning data memory 53.

[0355] With the NR circuit shown in FIG. 10, the input reliabilityα_(x(t)) is calculated following Expression (22) at the inputreliability calculating unit 61, from the parameter control data a₁through a_(N) obtained as described above.

[0356] Accordingly, in this case as well, learning is performed of theparameter control data a₁ through a_(N) stipulating the inputreliability α_(x(t)) in Expression (22), based on learning operatingsignals supplied according to operations of the user, so the operationsmade by the user can be learned without the user knowing, and further,optimal processing for the user can be performed using the results oflearning.

[0357] Also, with the NR circuit in FIG. 10, as with the NR circuit inFIG. 4, upon the user operating the operating unit 2, the operatingsignal processing unit 50 outputs weight representing operating signalscorresponding to the operating thereof, and the selecting unit 41selects the weight and supplies this to the correcting unit 21. In thiscase, the correction processing represented with the Expression (8) isperformed at the correcting unit 21, using the weight corresponding tothe user operations. In the event that the weight w(t) in Expression (8)is to be changed by user operations, the contents of the processing(correction processing) represented with Expression (8) is also changedas a matter of course, with the NR circuit in FIG. 10 as well, the “thecontents of processing” are changed following operations made by theuser so that output signals desired by the user are obtained.

[0358] Further, with the NR circuit in FIG. 10, in cases wherein theparameter control data a₁ through a_(N) cannot be obtained, or in caseswherein this can be obtained but the Expression (23) defined by theparameter control data a₁ through a_(N) does not precisely approximatethe relation between the student data and tutor data stored in thelearning data memory 53, weight obtained from input reliability based ondispersion of input signals is used for correction processing at thecorrecting unit 21. On the other hand, in the event that the parametercontrol data a₁ through a_(N) is obtained, and the Expression (23)defined by the parameter-control data a₁ through a_(N) preciselyapproximates the relation between the student data and tutor data storedin the learning data memory 53, the weight obtained from the inputsignals and the parameter control data a₁ through a_(N) (rather, theinput reliability calculated to have therefrom) and output reliabilityis used for correction processing at the correcting unit 21 followingthe Expression (23) defined by the parameter control data a₁ througha_(N) obtained by learning being performed using learning pairs obtainedbased on operations of operating unit 2 by the user.

[0359] That is, with the NR circuit in FIG. 10, as with the NR circuitin FIG. 4, algorithms for calculating weight to be used for correctionprocessing change between cases wherein sufficient numbers of learningpairs or learning pairs enabling the highly precise approximation arenot obtained, and cases wherein learning pairs enabling the highlyprecise approximation are obtained.

[0360] Accordingly, with the NR circuit in FIG. 10 as well, the“contents of processing”, and further, the “structure of processing”, isalso changed following user operations, whereby output signals desiredby the user are output.

[0361] Now, in the above case, the parameter control data a₁ througha_(N) is obtained using the output reliability α_(y(t−1)) as studentdata, with this output reliability α_(y(t−1)) being obtained from theinput reliability α_(x(t−1)), as shown in Expression (5). The inputreliability α_(x(t)) is gradually improved toward obtaining of theweight desired by the user, by the control data learning processingshown in FIG. 13 being performed, so the output reliability α_(y(t−1))is also improved along with this.

[0362] Also, with the above-described case, the output reliability isset at a known value, and the input reliability is stipulated by theparameter control data a₁ through a_(N), thereby obtaining parametercontrol data a₁ through a_(N) which can yield the weight desired by theuser, but conversely, an arrangement may be made wherein the inputreliability is set at a known value, and the output reliability isstipulated by the parameter control data a₁ through a_(N), herebyobtaining parameter control data a₁ through a_(N) which can yield theweight desired by the user.

[0363] Further, an arrangement may be made wherein, for example, theoutput reliability is set at a known value, the input reliability isstipulated by the parameter control data a₁ through a_(N), wherebyparameter control data a₁ through a_(N) yielding weight desired by theuser can be obtained, the input reliability obtained by the parametercontrol data a₁ through a_(N) is further made to be a known value, andthe output reliability is stipulated by parameter control data a₁′through a_(N)′, whereby parameter control data a₁ through a_(N) yieldingweight desired by the user can be obtained, i.e., two sets of parametercontrol data a₁ through a_(N) and a₁′ through a_(N)′ can be obtained.

[0364] Also, in the above-described case, description has been made withthe input reliability α_(x(t)) and the output reliability α_(y(t−1))being defined to obtain the parameter control data a₁ through a_(N), asshown in Expression (6), but besides this, an arrangement can be madesuch as, for example, shown in Expression (31) wherein the weight isdefined using, besides the input reliability α_(x(t)) and the outputreliability α_(y(t−1)), a correction term Δα of the input reliabilityα_(x(t)) and the output reliability α_(y(t−1)), so as to enableobtaining the parameter control data a₁ through a_(N) and the correctionterm Δα. $\begin{matrix}{\text{[Expression~~31]}{{w(t)} = \frac{\alpha_{y{({t - 1})}} + {\Delta\alpha}}{\alpha_{y{({t - 1})}} + {\Delta\alpha} + \alpha_{x{(t)}}}}} & (31)\end{matrix}$

[0365] Further, the expression for defining the input reliability by theparameter control data is not restricted to the Expression (22).

[0366] Further, FIG. 14 illustrates a second detailed configurationexample of the optimizing device shown in FIG. 1. The optimizing device1 shown in FIG. 14 is configured with an internal information generatingunit 71 newly provided therein, while the configuration of theprocessing unit 11 is the same as that in FIG. 2 and accordinglydescription thereof will be omitted. Also, a display unit 8 is providedexternally to the optimizing device 1 with the embodiment shown in FIG.14.

[0367] The internal information generating unit 71 reads out internalinformation of the processing unit 11, and converts this into imageinformation, which is then output and displayed (presented) to thedisplay unit 81 made up of an LCD (Liquid Crystal Display) or CRT(Cathode Ray Tube). More specifically, the display unit 81 may bearranged to display the internal information as it is, numerically, ormay make a display wherein a level gauge like display screen is set, andthe level gauge fluctuates according to values of the internalinformation. Also, the display unit 81 is not restricted to this, andmay be any other display method so long as internal information isdisplayed (presented) visually. Now, for internal information, theweight stored in the weight memory 31 and 32 of the correcting unit 21,the stored contents of the learning data memory 53 or learninginformation memory 55 of the learning unit 22, and so forth, can beemployed. Also, the internal information may be presented to the userwith presenting (providing) methods other than displaying, i.e., bysound or the like.

[0368] Next, the optimizing processing of the NR circuit shown in FIG.14 will be described with reference to the flowchart shown in FIG. 15.This processing is basically the same as the optimizing processingdescribed with reference to the flowchart in FIG. 3, and differs in thepoint that processing for displaying internal information from theinternal information generating unit 71 has been added. That is to say,the same processing as that in steps S1 through S11 is performed insteps S91 through S101, and the flow proceeds to step S102.

[0369] In step S102, the weight W is displayed on the display unit 81.That is, the internal information generating unit 71 reads out the valueof the weight W stored in the weight memory 31, for example, convertsthis into image signals which can be displayed on the display unit 81and outputs to the display unit 81, thereby displaying (presenting) theweight W, and the flow returns to step S91.

[0370] Due to the processing described with reference to the flowchartshown in FIG. 15, the weight W serving as internal information relatingto the processing which is actually being executed in the processingunit 11 is displayed (presented) to the user, and consequently, the usercan operate the operating unit 2 so as to obtain optimal output signalswhile viewing the display of the internal information. Note that, withthe internal information generating unit 71, in addition to theabove-described internal information such as weight, the parametercontrol data a and b may be read out from the parameter control datamemory 37 (FIG. 4, FIG. 10) in the learning unit 22 and is displayed,for example. Also, an arrangement may be made to generate imageinformation, as internal information, representing the weight selectedby the selecting unit 41 (FIG. 4, FIG. 10) is a weight obtained from theparameter control data a and b obtained by learning being performedusing learning pairs, or weight obtained from the input reliability andoutput reliability.

[0371] Next, FIG. 16 illustrates the configuration example of anembodiment of an auto-driving device for an automobile, to which theoptimizing device 1 shown in FIG. 1 has been applied.

[0372] With the auto-driving device, the location coordinates (X, Y) ofthe automobile and the direction of driving θ are obtained, and theautomobile is driven following a predetermined course. However, errorsare often contained in the location coordinates (X, Y) and the directionof driving θ obtained with the auto-driving device, and in this case,the automobile may run off of the predetermined course. Accordingly,with the auto-driving device shown in FIG. 16, the operations of theuser are learned without the user knowing, and the automobile is causedto drive along the predetermined course, based on the results oflearning. That is, in the event that the automobile runs off thepredetermined course, generally, the user will operate the steeringwheel or gas pedal to cause the automobile to run along thepredetermined course. Accordingly, the auto-driving device shown in FIG.16 effects control such that the operations of the user are learnedwithout the user knowing, and the automobile is caused to gradually cometo drive along the predetermined course, based on the results oflearning.

[0373] A gyro sensor 91 detects a yaw rate r of the automobile, andsupplies this to a computing unit 93. A wheel pulser 92 supplieselectric pulses of a number corresponding to the rotational angle of thewheels of the automobile, to the computing unit 93.

[0374] The computing unit 93 computes the location coordinates (X, Y) ofthe automobile and the direction of driving θ from the output of thegyro sensor 91 and wheel pulser 92, following the following expression,for example, and supplies this to the optimizing device 94.

[0375] [Expression 32]

θ=θ(0)+∫rdt

X=X(0)+∫V _(r) cos(θ+β)dt

Y=Y(0)+∫V _(r) sin(θ+β)dt  (32)

[0376] Note that in Expression (32), θ(0) represents the direction ofthe automobile at the time of starting driving, and (X(0), Y(0))represent the coordinates of the automobile at the time of startingdriving. Also, θ(0) and (X(0), Y(0)) may be obtained using an unshownGPS (Global Positioning System) or the like. Further, V_(r) representsthe driving speed of the automobile, and β represents the slip angle ofthe center of gravity of the automobile.

[0377] Now, how to obtain the coordinates (X, Y) of the automobile anddriving direction θ as described above is disclosed in, for example,Japanese Unexamined Patent Application Publication No. 10-69219.

[0378] The optimizing device 94 is configured of a processing unit 101,learns the operations which the user makes with an operating unit 98,i.e., performs learning based on operating signals supplied by the useroperating the operating unit 98, corrects the coordinates (X, Y) anddriving direction θ from the computing unit 93 so that driving which theuser desires is performed, based on the results of learning, andsupplies this to the auto-driving control unit 95.

[0379] The auto-driving control unit 95 stores map data and a coursewhich has been set beforehand for auto-driving (hereafter may bereferred to as a set course). The auto-driving control unit 95recognizes the current position and driving direction of the automobilefrom the coordinates (X, Y) and driving direction θ supplied from theprocessing unit 101 of the optimizing device 94, and generates controlsignals for controlling a later-described driving unit 97, so that theautomobile follows the set course for driving, and outputs this to theselecting unit 96.

[0380] The selecting unit 96 is supplied with control signals from theauto-driving control unit 95, and is also supplied with operatingsignals from the operating unit 98. Of the control signals from theauto-driving control unit 95 and the operating signals from theoperating unit 98. The selecting unit 96 selects the operating signalswith higher priority, and outputs these to the driving unit 97, that is,the selecting unit 96 normally selects the control signals from theauto-driving control unit 95 and outputs these to the driving unit 97,but upon receiving operating signals from the operating unit 98, stopsoutput of the control signals from the auto-driving control unit 95while operating signals are being received, and outputs the operatingsignals from the operating unit 98 to the driving unit 97.

[0381] The driving unit 97 follows control signals or operating signalsfrom the selecting unit 96, and drives the mechanisms of the automobilewhich are necessary for driving, such as an unshown engine, wheels,brakes, clutch, etc. The operating unit 98 is configured of, forexample, the steering wheel, gas pedal, brake pedal, clutch pedal, etc.,and supplies operating signals corresponding to operations made by theuser, to the optimizing device 94 and selecting unit 96.

[0382] With the auto-driving device configured as described above, atthe computing unit 93, the current coordinates (X, Y) and drivingdirection θ of the automobile are computed from the output of the gyrosensor 91 and wheel pulser 92, and supplied to the auto-driving controlunit 95 via the processing unit 101 of the optimizing device 94. Theauto-driving control unit 95 recognizes the current position and drivingdirection of the automobile from the coordinates (X, Y) and drivingdirection θ supplied thereto, generates control signals for controllingthe later-described driving unit 97 so that the automobile drivesfollowing the set course, and supplies these to the driving unit 97 viathe selecting unit 96. Thus, the automobile performs auto-drivingfollowing the control signals which the auto-driving control unit 95outputs.

[0383] On the other hand, upon the user operating the operating unit 98,operating signals corresponding to the operation thereof are supplied tothe driving unit 97 via the selecting unit 96, whereby the automobiledrives following the operations which the user makes at the operatingunit 98.

[0384] Further, due to the user operating the operating unit 98,operating signals which operating unit 98 outputs are supplied to theprocessing unit 101 of the optimizing device 94 as well. The optimizingdevice 94 performs learning based on operating signals supplied by theuser operating the operating unit 98. Upon the user stopping theoperations of the operating unit 98, the processing unit 101 ofoptimizing device 94 then makes corrections to the coordinates (X, Y)and driving direction θ supplied from the computing unit 93 so thatdriving is performed following the set course as the driving which theuser desires, based on the results of learning, and supplies this to theauto-driving control unit 95.

[0385] Next, FIG. 17 illustrates a configuration example of theprocessing unit 101 of the optimizing device 94 in FIG. 16. In thefigure, the parts corresponding to the processing unit 11 shown in FIG.4 are denoted with the same reference numerals, and description thereofwill be omitted where suitable in the following. That is, the processingunit 101 shown in FIG. 17 is configured basically the same as theprocessing unit 11 shown in FIG. 4 except that no selecting unit 41 isprovided, and an operating signal processing unit 110 and tutor datagenerating unit 111 are provided instead of the operating signalprocessing unit 50 and tutor data generating unit 51, respectively.

[0386] Now, to facilitate description in the following, of thecoordinates (X, Y) and driving direction θ supplied from the computingunit 93 to the processing unit 101 of the optimizing device 94,description will be made giving consideration to the driving direction θalone. However, the coordinates (X, Y) may be subjected to the sameprocessing as the driving direction θ to be described now.

[0387] The operating signal processing unit 110 receives operatingsignals from the operating unit 98, and judges whether or not these arelearning operating signals. In the event the operating signals arelearning operating signals, the operating signal processing unit 110then supplies a message to that effect to the student data generatingunit 52 and tutor data generating unit 111.

[0388] The tutor data generating unit 111 is arranged so as to besupplied with messages from the operating signal processing unit 110 tothe effect that the operating signals are learning operating signals(hereafter may be referred to as learning messages), and also so as tobe supplied with the driving direction θ from the computing unit 93,serving as input signals. Further, the driving direction θ from thecomputing unit 93 which has been corrected (hereafter may be referred toas corrected driving direction), is also supplied to the tutor datagenerating unit 111 as output signals output from the correcting unit 21(computing device 36). The tutor data generating unit 111 obtains theweight W corresponding to the learning operating signals, from thedriving direction θ serving as input signals supplied at the time ofreceiving the learning message, and the corrected driving directionserving as output signals, which is supplied to the learning data memory53 as tutor data.

[0389] That is, in this case, it is necessary to obtain, as tutor data,the weight W at the point of the automobile heading in a predetermineddirection following the user operating the operating unit 98 which isthe steering wheel so that the automobile heads in a predetermineddirection. That is, it is necessary to employ as tutor data the weight Wused for correction of the input signals x(t) indicating the drivingdirection θ, immediately following the user operating the operating unit98 serving as the steering wheel and the automobile heading in thepredetermined direction. The input signals x(t) immediately followingoperation of the operating unit 98 are corrected, following Expression(8), into output signals y(t) immediately following operating theoperating unit 98 by weighted addition of the input signals x(t) and theoutput signals y(t−1) output immediately before operating the operatingunit 98, so the weight W used for correction of the input signals x(t)immediately following operation of the operating unit 98 can be obtainedfrom Expression (8), from the input signals x(t) immediately followingoperation of the operating unit 98, and output signals y(t) immediatelyfollowing operating the operating unit 98 and output signals y(t−1)output immediately before operating the operating unit 98. Accordingly,the tutor data generating unit 111 obtains the weight W from the drivingdirection θ serving as the input signals x(t) supplied immediatelyfollowing a reception of the learning message, and the corrected drivingdirection which is the output signals y(t−1) and y(t) suppliedimmediately before and immediately following reception of the learningmessage, respectively, and supplies this to the learning data memory 53.

[0390] Also, upon receiving the learning message, the student datagenerating unit 52 supplies the weight w obtained from the drivingdirection which is the input signals supplied to immediately before thatpoint, to the learning data memory 53, as student data. That is, asdescribed with reference to FIG. 4, the student data generating unit 52is configured in the same way as the input reliability calculating unit42, output reliability calculating unit 43, latch circuit 44, and weightcalculating unit 45, and calculates the weight w of the drivingdirection which is the input signals supplied thereto (the same weight wthat is obtained at the weight calculating unit 45), and supplies theweight w calculated immediately before receiving the learning message tothe learning data memory 53, as student data.

[0391] Accordingly, at the parameter control data calculating unit 54,the weight W at the point that the user operates the operating unit 98and the driving direction becomes the direction which the user desiresis taken as tutor data, and the same weight w as that which the weightcalculating unit 45 output immediately before the user operating theoperating unit 98 is taken as student data, and the parameter controldata a and b shown in the Expressions (20) and (21) is performed.

[0392] Then, at the weight correcting unit 46, the weight w obtained atthe weight calculating unit 45 is corrected following Expression (13),using the parameter control data a and b, and this is supplied to thecorrecting unit 21.

[0393] As a result, the parameter control data a and b serve to correctthe weight w obtained at the weight calculating unit 45 so as to correctthe driving direction immediately before the user operating theoperating unit 98 to the driving direction immediately following theuser operating the operating unit 98, so the automobile performsauto-driving along the set course.

[0394] That is, it can be considered that the fact that the useroperates the operating unit 98 means that the driving direction θ whichthe computing unit 93 outputs contains error due to error in the gyrosensor 91 or noise in the output thereof, computation error in thecomputing unit 93, etc., and accordingly does not represent the truedriving direction of the automobile, and accordingly the actual drivingdirection of the automobile has deviated from the set course. Further,it can be considered that the operations of the operating unit 98 madeby the end user in this case are for changing the actual drivingdirection of the automobile to that following the set course.Accordingly, performing learning wherein the weight W at the point thatthe user operates operating unit 98 and the actual driving direction ofthe automobile follows the set course is taken as tutor data, and theweight w obtained at the weight calculating unit 45 immediately beforethe user operates the operating unit 98, i.e., the weight w which theweight calculating unit 45 outputs in the state of deviating from theset course, as student data, the parameter control data a and b of theExpression (13) for correcting the weight in the Expression (6), so asto correct the driving direction in the state of deviating from the setcourse to a direction following the set course.

[0395] Next, description will be made regarding the processing unit 101of the optimizing device 94 shown in FIG. 17. Now, as with theprocessing unit 11 of the NR circuit shown in FIG. 4, the processingunit 101 of the optimizing device 94 shown in FIG. 17 performs controldata learning processing for obtaining the parameter control data forcontrolling (correcting) the weight serving as correction parameters, bycorrection processing for correcting the driving direction θ which thecomputing unit 93 outputs as input signals x(t), correction parametercomputing processing for obtaining the weight serving as correctionparameter is to be used for the correction processing thereof, andoperations of the operating unit 98 (FIG. 16) by the user, but thecorrection processing is the same as the correction processing performedby the NR circuit shown in FIG. 4 that has been described with referenceto FIG. 7, so here, the correction parameter computing processing andstudent data learning processing which the processing unit 101 of theoptimizing device 94 shown in FIG. 17 performs, will be described.

[0396] First, the correction parameter computing processing which theprocessing unit 101 of the optimizing device 94 shown in FIG. 17performs will be described with reference to the flowchart shown in FIG.18.

[0397] In the correction parameter computing processing, first, in stepS11, the input signals calculating unit 22 obtains the input reliabilityαx(t) based on dispersion of the driving direction θ from the computingunit 93 (FIG. 16) serving as input signals, and supplies this to theoutput reliability calculating unit 43 and weight calculating unit 45,in the same way as with step S31 in FIG. 7.

[0398] Subsequently, the flow proceeds to step S112, where the weightcalculating unit 45 uses the input reliability x(t) from the inputreliability calculating unit 42 to obtain the weight w(t) which issupplied to the weight correcting unit 46, following Expression (6), andthe flow proceeds to step S113.

[0399] In step S113, the weight correcting unit 46 reads out theparameter control data from the parameter control data memory 57, andthe flow proceeds to step S114. In step S114, the weight correcting unit46 judges whether or not the parameter control data read out from theparameter control data memory 57 is auto mode data which represents amode wherein the weight w(t) is not corrected, i.e., a mode wherein aweight w(t) which is obtained automatically, so as to say, from theinput reliability and output reliability at the weight calculating unit45, as the weight W for correcting the input signals x(t) without anychange, regardless of operations of operating unit 98 (FIG. 16) by theuser (auto mode).

[0400] In the event that judgment is made in step S113 that theparameter control data is not auto mode data, the flow proceeds to stepS115, where the weight correcting unit 46 corrects the weight w(t)supplied from the weight calculating unit 45 following the linearexpression in the Expression (13) defined by the parameter control dataa and b supplied from the parameter control data memory 57, and the flowproceeds to step S116. In step S116, weight correcting unit 46 suppliesthe post-correction weight to the correcting unit 21 as correctionparameters, and the flow proceeds to step S117.

[0401] On the other hand, in the event that judgment is made in stepS114 that the parameter control data is auto mode data, step S115 isskipped, the flow proceeds to step S116 where the weight correcting unit46 supplies the weight w(t) from the weight calculating unit 45 withoutchange to the correcting unit 21 as correction parameters, and the flowproceeds to step S117.

[0402] At step S117, the output reliability calculating unit 43 updatesthe output reliability. That is, the output reliability calculating unit43 adds the input reliability α_(x(t)) calculated by input reliabilitycalculating unit 42 immediately preceding step S31 and the outputreliability α_(y(t−1)) from one sample back which the latch circuit 44has latched, following Expression (5), thereby obtaining the currentoutput reliability α_(y(t)), which is stored in the form of overwritingon the latch circuit 44.

[0403] Following the processing in step S117, the flow returns to stepS111, and subsequently the same processing is repeated.

[0404] As described above, with the correction parameter computingprocessing shown in FIG. 18, in the event of being in a auto mode, theweight to be used for correction processing is obtained from the inputreliability and output reliability regardless of the operations of theoperating unit 98, and in the event of not being in the auto mode, theweight to be used for correction processing is obtained using theparameter control data obtained by the learning in the control datalearning processing in FIG. 19 which will be described later, based onthe operations of the operating unit 98.

[0405] Next, description will be made regarding the control datalearning processing which the processing unit 101 of the optimizingdevice 94 shown in FIG. 17 performs, with reference to the flow chartshown in FIG. 19.

[0406] With the control data learning processing, first, in step S131,the operating signal processing unit 110 judges whether or not learningoperating signals have been received from the operating unit 98 (FIG.16), and in the event that judgment is made that the signals have notbeen received, the flow returns to step S101.

[0407] Also, in the event that judgment is made in step S101 thatlearning operating signals have been received from the operating unit98, i.e., in the event of that, for example, the steering wheel or thelike which is the operating unit 98 starts to be operated, followingwhich, without elapsing of a first time t1 or longer, operating isperformed continuously for a second time t2 or longer, following whichthe operations thereof are stopped continuously for a third time t3 orlonger, or cases wherein following starting the operations of thesteering wheel which is the operating unit 98, operations thereof arestopped continuously for a third time t3 or longer, such that judgmentcan be made that the user has operated the steering wheel which isoperating unit 98 so as to direct the automobile in the desireddirection, the flow proceeds to step S132, where the tutor datagenerating unit 111 generates tutor data, and the student datagenerating unit 52 generates student data.

[0408] That is to say, upon judging that the learning operating signalshave been received, the operating signal processing unit 110 supplies alearning message to the tutor data generating unit 111 and student datagenerating unit 52. Upon receiving the learning message from theoperating signal processing unit 110, in step S132 the tutor datagenerating unit 111 obtains a weight w corresponding to the learningoperating signals, from the driving direction 0 which is the inputsignals supplied from the computing unit 93, and the corrected drivingdirection 0 from the computing unit 93 (corrected driving direction)which is the output signals output by the correcting unit 21 (computingdevice 36).

[0409] Specifically, the tutor data generating unit 111 receives inputsignals x(t) representing the driving direction θ immediately followingthe user operating the operating unit 98 serving as the steering wheeland the automobile heading in the desired direction, from the computingunit 93 (FIG. 16). Further, the tutor data generating unit 111 isarranged so as to hold the current output signals y(t) output by thecorrecting unit 21, and output signals y(t−1) from one timing back,i.e., the output signals y(t−1) immediately before operating theoperating unit 98, and obtains the weight W used at the correcting unit21 at the time that the learning operating signals are provided to (theweight corresponding to the learning operating signals) followingExpression (8), using the input signals x(t), and output signals y(t)and y(t−1).

[0410] Note that in this case, operations of the steering wheel which isthe operating unit 98 made by the user are considered to beinstantaneously completed within one timing from t−1 to t, to facilitatedescription.

[0411] The tutor data generating unit 111 obtains the weight Wcorresponding to the learning operating signals as described above, andthen supplies the weight W to the learning data memory 53 as tutor data.

[0412] Further, in step S132, the student data generating unit 52 whichhas received the learning message from the operating signal processingunit 110 supplies the same weight w as that output by the weightcalculating unit 45, calculated using the input reliability and outputreliability obtained from the driving direction which is input signalssupplied from the computer unit 93 (FIG. 16), to the learning datamemory 53, as student data.

[0413] Accordingly, the learning data memory 33 is supplied with alearning pair, wherein the weight W used at the correcting unit 21 atthe point that the actual driving direction of the automobile becomesthe direction which the user desires by the user operating the operatingunit 98, is taken as tutor data, and the weight w obtained from theinput reliability and output reliability immediately before the useroperating the operating unit 98, is taken as student data.

[0414] Upon receiving the tutor data W from the tutor data generatingunit 111 and the student data w from the student data generating unit52, the learning data memory 53 stores the set of the newest tutor dataW and student data w in step S133, and the flow proceeds to step S134.

[0415] In step S134, the parameter control data calculating unit 54performs adding in the least-square, with regard to the newest tutordata and student data stored in the learning data memory 53 and to thelearning information stored in the learning information memory 55, inthe same way as with step S44 in FIG. 8. Further, in step S134, theparameter control data calculating unit 54 stores the adding-in resultsas learning information, in the form of overwriting the learninginformation memory 55, and the flow proceeds to step S135.

[0416] In step S135, the parameter control data calculating unit 54judges whether or not the parameter control data a and b can be obtainedwith the Expressions (20) and (21), from the adding-in results servingas the learning information stored in the learning information memory55, as with the case in step S45 in FIG. 8.

[0417] In the event that judgment is made in step S135 that theparameter control data a and b cannot be obtained, the parameter controldata calculating unit 54 supplies the judgment control unit 56 withinformation to that effect, and the flow proceeds to step S139. In stepS139, the judgment control unit 56 supplies the auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored. Then, the flow returns tostep S131, and subsequently, the same processing is repeated.

[0418] Accordingly, in the event that sufficient learning informationwhereby the parameter control data a and b can be obtained does notexist, the weight w(t) automatically obtained from the input reliabilityand output reliability at the weight calculating unit 45 (FIG. 17) isused for correction of input signals x(t) without change.

[0419] On the other hand, in the event that judgment is made in stepS135 that the parameter control data a and b can be obtained, the flowproceeds to step S136, where the parameter control data calculating unit54 uses learning information to calculate the Expressions (20) and (21),and thereby obtaining the parameter control data a and b, which issupplied to the judgment control unit 56, and the flow proceeds to stepS137.

[0420] In step S137, the judgment control unit 56 follows the linearexpression in Expression (13) defined by the parameter control data aand b from the parameter control data calculating unit 54 to obtainprediction values of a corresponding tutor data from the student datastored in the learning data memory 53, and obtains the sum of thesquare-error represented by the Expression (15) for the prediction errorof that prediction value (error as to the tutor data stored in thelearning data memory 53). Further, the judgment control unit 56 obtainsa normalization error wherein the square-error sum is divided by thenumber of the learning pairs stored in the learning data memory 53, forexample, and the flow proceeds to step S138.

[0421] In step S138, the judgment control unit 56 judges whether or notthe normalization error is greater than (equal to or more than) apredetermined threshold value S1. In the event that judgment is made instep S138 that the normalization error is greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression in theExpression (13) defined by the parameter control data a and b does notprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the flow proceeds tostep S139 where the judgment control unit 56 supplies auto mode datarepresenting the auto mode to the parameter control data memory 57 asparameter control data, where it is stored, as described above. The flowthen returns to step S131, and subsequently, the same processing isrepeated.

[0422] Accordingly, even in the event that the parameter control data aand b can be obtained, in the event that the linear expression in theExpression (13) defined by the parameter control data a and b does notprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the weight w(t)automatically obtained from the input reliability and output reliabilityis used for correcting the input signals x(t), in the same way as caseswherein sufficient learning information for obtaining the parametercontrol data a and b does not exist.

[0423] On the other hand, in the event that judgment is made in stepS138 that the normalization error is not greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression in theExpression (13) defined by the parameter control data a and b doesprecisely approximate the relation between the student data and thetutor data stored in the learning data memory 53, the flow proceeds tostep S140, wherein the judgment control unit 56 obtains the error(distance) ε between the regression line represented by the linearexpression in the Expression (13) defined by the parameter control dataa and b from the parameter control data calculating unit 54 and a pointstipulated by the newest tutor data and student data stored in thelearning data memory 53.

[0424] The flow then proceeds to step S141, where the judgment controlunit 56 judges whether or not the magnitude of the error ε is greaterthan (equal to or more than) a predetermined threshold value S2, and inthe event that judgment is made that this is not greater, step S142 isskipped, the flow proceeds to step S143, and the judgment control unit56 outputs the parameter control data a and b obtained in step S136 tothe parameter control data memory 37. The parameter data memory 57stores the parameter control data a and b from the judgment control unit56 in the form of overwriting, and the flow returns to step S131 on theother hand, in the event that judgment is made in step S141 that themagnitude of the error ε is greater than the predetermined thresholdvalue S2, the flow proceeds to step S142, where the judgment controlunit 56 controls the parameter control data calculating unit 54 tore-calculate the parameter control data a and b using only apredetermined number of the past learning pairs of from the newestlearning pairs of newest tutor data and student data stored in thelearning data memory 53 (without using the learning information in thelearning information memory 55) by controlling the parameter controldata calculating unit 54. The flow then proceeds to step S143, where thejudgment control unit 56 outputs the parameter control data a and bobtained in step S142 to the parameter control data memory 37, where itis stored in the form of overwriting, and the flow returns to step S131.

[0425] Accordingly, in the event that the parameter control data a and bcan be obtained, and linear expression of the Expression (13) defined bythe parameter control data a and b precisely approximates the relationbetween the student data and tutor data stored in the learning datamemory 53, the weight w(t) obtained from the input reliability andoutput reliability is corrected following the Expression (13) defined bythe parameter control data a and b obtained by the learning beingperformed using the learning pairs obtained based on the operations ofthe operating unit 2 by the user, and the corrected weight W obtained bythe correction is used for correcting the input signals x(t).

[0426] As described above, with the auto-driving device shown in FIG. 16as well, judgment is made regarding whether or not the operating signalssupplied in response to operations of the user can be used for learning,and in the event that these are learning operating signals which can beused for learning, the parameter control data a and b for correcting theweight for correcting the input signals is learned based on the learningoperating signals, so the operations of the user can be learned withoutthe user knowing, and consequently, processing suitable for the usergradually comes to be performed based on the results of learning, andultimately, processing optimal for the user is performed.

[0427] That is, as the user performs operations with operating unit 98so as to correct the driving direction to follow the set course, theautomobile gradually comes to perform auto-driving following the setcourse.

[0428] Also, in the same manner as the NR circuit shown in FIG. 4, withthe processing unit 101 of the optimizing device 94 shown in FIG. 17 aswell, the weight W used for the correction processing (FIG. 6) performedat the correcting unit 21 is changed, following the operations of theuser at the operating unit 98, so that the actual driving direction ofthe automobile follows the set course. That is to say, upon the useroperating the operating unit 98 so that the driving direction of theautomobile heads in the desired direction, the driving direction θ whichis input signals output by the computing unit 93 (FIG. 16) changes, sothe input reliability obtained from the driving direction θ, and furtherthe output reliability obtained from the input reliability, change. Thechange in this input reliability and output reliability causes theweight obtained at the weight calculating unit 45 to change, and thischanged weight is supplied to the correcting unit 21 through the weightcorrecting unit 46. The correction processing represented by Expression(8) is then performed at the correcting unit 21, using the weight thathas been supplied. Accordingly, in the event that the user operates theoperating unit 98, the weight of the Expression (8) is changed by theoperations of the user, and then the same manner as the case describedwith the NR circuit in FIG. 4, the contents of the processing(correction processing) represented by Expression (8) also change as amatter of course, so it can be said that the “contents of processing”are being changed so that the driving direction desired by the user canbe obtained, following operations of the user, with the processing unit101 of the optimizing device 94 shown in FIG. 17, as well.

[0429] Further, as with the NR circuit in FIG. 4, with the processingunit 101 of the optimizing device 94 shown in FIG. 17 as well, in caseswherein sufficient learning pairs have not been input from the user, orin the event that learning pairs whereby highly precise approximation ispossible have not been input, weight automatically obtained from theinput reliability and output reliability is used for the correctionprocessing at the correcting unit 21, and in the event that learningpairs whereby highly precise approximation is possible have been input,correction weighting obtained by the parameter control data a and bobtained by learning being performed using the learning pairs is usedfor correction processing at the correcting unit 21. That is to say, thealgorithms for calculating weight used for the correction processingchange depending on the cases wherein a sufficient number of learningpairs or learning pairs enabling highly precise approximation are notobtained, and cases wherein learning pairs enabling highly preciseapproximation are obtained.

[0430] Accordingly, as with the NR circuit in FIG. 4, with theprocessing unit 101 of the optimizing device 94 in FIG. 17 as well, the“contents of processing”, and further, the “structure of processing”, isalso changed following user operations, whereby the automobile performsauto-driving in the driving direction following the set course.

[0431] Also, for example, Japanese Unexamined Patent ApplicationPublication No. 7-13625 discloses that driving control device for workvehicles such as rice transplanters and the like, and with the drivingcontrol device, the amount of correction of control parameters in theautomatic steering state is computed so that the difference between thestate of operation by the user and the information based on detectionresults such as of the gyro sensor and the like, are reduced.Accordingly, the auto driving device shown in FIG. 16 holds in commonwith the driving control device described in Japanese Unexamined PatentApplication Publication No. 7-13625, the point that the amount ofcorrection of parameters for auto driving (automatic steering) changesbased on operations made by the user.

[0432] However, the auto driving device shown in FIG. 16 greatly differsfrom the driving control device described in Japanese Unexamined PatentApplication Publication No. 7-13625 which computes the amount ofcorrection of control parameters in the auto steering state only in theevent that a switch is manually switched over to a manual steeringcontrol mode, with regard to the point of judging whether or notoperating signals supplied according to operations made by the user canbe used for learning, and the point that, in the event that theoperating signals are learning operating signals which can be used forlearning, parameter control data for correcting the weighting forcorrecting input signals is learned based on the learning operatingsignals.

[0433] Due to such differences, with the driving control devicedescribed in Japanese Unexamined Patent Application Publication No.7-13625, every time the user feels that suitable automatic steering isnot being performed, the user must switch over a switch to a manualsteering control mode, and following computation of the amount ofcorrecting the control parameters, switch over the switch again to theauto steering control mode, which may leave the user with an impressionof being troublesome.

[0434] Conversely, with the auto driving device shown in FIG. 16, thejudgment is made regarding whether or not operating signals suppliedaccording to operations made by the user can be used for learning, andfurther, in the event that these are learning operating signals whichcan be used for learning, the algorithms are changed such that parametercontrol data for correcting the weighting for correcting input signalsis learned based on the learning operating signals, so suitable autodriving is performed even without the user performing such switchingover of the switch, as described above. That is to say, learning ofoperations by the user is performed without the user knowing, solearning proceeds as the user corrects the driving direction, and theautomobile gradually comes to drive following the set course, evenwithout the user making operations.

[0435] Further, with the auto driving device shown in FIG. 16, thestructure of processing changes corresponding to user operations, thusdiffering from the driving control device described in JapaneseUnexamined Patent Application Publication No. 7-13625 with regard tothis point as well.

[0436] Next, FIG. 20 illustrates another configuration example of theprocessing unit 101 of the optimizing device 94 shown in FIG. 16. Notethat the parts corresponding with the case shown in FIG. 17 in thefigure are denoted with the same reference numerals, and descriptionthereof will be omitted as appropriate in the following.

[0437] With the processing unit 11 of the NR circuit shown in FIG. 4 andFIG. 10, and the processing unit 101 of the optimizing device 94 shownin FIG. 17, parameter control data for controlling correction parametersare learned using learning pairs obtained based on operations made bythe user, but with the processing unit 101 shown in FIG. 20, correctionparameters themselves are learned using the learning pairs obtainedbased on operations made by the user.

[0438] That is, with the embodiment shown in FIG. 20, the correctingunit 21 is configured of a correction amount calculating unit 141 andcomputing device 142, and the learning unit 22 is configured of learningdata memory 53, learning information memory 55, judgment control unit56, operating signal processing unit 110, tutor data generating unit143, student data generating unit 144, correction parameter calculatingunit 145, and correction parameter memory 146.

[0439] The later-described correction parameters are supplied to thecorrection amount calculating unit 141 from the correction parametermemory 146 in the learning unit 22, with the correction amountcalculating unit 141 using the correction parameters to calculate thecorrection amount for correcting the driving direction 0 which is theinput signals, and supplying this to the computing device 142.

[0440] The computing device 142 is supplied with correction amount fromthe correction amount calculating unit 141, and is also supplied withthe driving direction θ which is the input signals, from the computingunit 93 (FIG. 16), with the computing unit 142 correcting the drivingdirection 6 which is the input signals by adding the correction amountthereto, and outputting the post-correction driving direction (correcteddriving direction) to the auto driving control unit 95 (FIG. 16) asoutput signals.

[0441] The tutor data generating unit 143 supplies the driving directionwhich is the input signals supplied immediately following reception ofthe learning message from the operating signal processing unit 110, tothe learning data memory 53, as tutor data. The student data generatingunit 144 supplies the driving direction which is the input signalssupplied immediately before reception of the learning message from theoperating signal processing unit 130, to the learning data memory 53, asstudent data.

[0442] The correction parameter calculating unit 145 learns correctionparameter which minimizes a predetermined statistical error by computingnew learning information, using tutor data and student data serving aslearning data stored in the learning data memory 53, and further, ifnecessary, using learning information stored in learning informationmemory 55, and supplies this to the judgment control unit 56 undercontrol of the judgment control unit 56. The correction parametercalculating unit 145 updates the stored contents of the learninginformation memory 55, with the new learning information obtained bylearning.

[0443] The correction parameter memory 146 stores correction parameterswhich the judgment control unit 56 outputs.

[0444] With optimizing device 94 configured as described above, thedriving direction θ supplied from the computing unit 93 is corrected asdescribed next.

[0445] That is, with the yaw rate output by the gyro sensor 91 (FIG. 16)at a time t as r′, the driving direction is calculated at the computingunit 93 from an expression wherein r in Expression (32) is substitutedwith r′.

[0446] Now, with the error contained in the yaw rate r′ output by thegyro sensor 91 as e_(r), and the true yaw rate as r, the yaw rate r′output by the gyro sensor 91 is represented by the following expression.

[0447] [Expression 33]

r′=r+e _(r)  (30)

[0448] At the computing unit 93, the driving direction θ′ calculatedfrom the yaw rate r′ output from the gyro sensor 91 is as follows, fromthe Expressions (32) and (33).

[0449] [Expression 34] $\begin{matrix}\begin{matrix}{\theta^{\prime} = {{\theta (0)} + {\int{r^{\prime}{t}}}}} \\{= {{\theta (0)} + {\int{\left( {r + e_{r}} \right){t}}}}} \\{= {{\theta (0)} + {\int{r{t}}} + {\int{e_{r}{t}}}}}\end{matrix} & (34)\end{matrix}$

[0450] Accordingly, relation between the driving direction θ′ obtainedby the computing unit 93 and the true driving direction θ obtained fromthe true yaw rate r is as shown in the following expression.

[0451] [Expression 35]

θ′=θ+∫e _(r) dt  (35)

[0452] In the event that the error e_(r) contained in the yaw rate r′output by the gyro sensor 91 is white, the second term of the right-handside of Expression (35) is 0 in the long term, as shown in the followingexpression, so there is no particular problem. In the short term, thesecond term of the right-hand side of Expression (35) is not 0, but thiscase can be dealt with, with the processing unit 101 of the optimizingdevice 94 shown in FIG. 17.

[0453] [Expression 36]

∫e _(r) dt=0  (36)

[0454] However, in the event that the error e_(r) is colored, the errore_(r) accumulates as time t passes, so the driving direction θ′ obtainedfrom the computing unit 93 greatly deviates from the true drivingdirection θ.

[0455] That is, considering auto driving directly proceeding in aconstant direction in order to simplify description, control signalswhich make the driving direction θ′ obtained from the computing unit 93to be constant, as indicated by the dotted line in FIG. 21, aregenerated at the auto driving control unit 95 (FIG. 16).

[0456] However, in the event that the error e_(r) contained in thedriving direction θ′ obtained from the computing unit 93 is colored, theerror e_(r) accumulates as time t passes, so that the time that thedriving direction θ′ obtained from the computing unit 93 traces a coursecurving as indicated by the solid line in FIG. 21, for example, theautomobile is proceeding straight ahead.

[0457] Accordingly, with the processing unit 101 of the optimizingdevice 94 in FIG. 20, correction parameters learning processing isperformed for learning correction parameters a₀, a₁, . . . , a_(N), forcorrecting the driving direction θ′ based on learning operating signalsfrom the user, so that the driving direction θ′ obtained from thecomputing unit 93 supplied as input signals traces the course indicatedby the solid line in FIG. 21, and correction processing is performed forcorrecting the driving direction θ′ obtained from the computing unit 93using the correction parameters a₀ through a_(N).

[0458] Accordingly, the correction processing and correction parameterlearning processing which the processing unit 101 of the optimizingdevice 94 shown in FIG. 20 performs will be described with reference toFIG. 22 and FIG. 23. While the embodiment in FIG. 18 involves autodriving proceeding straight in a constant direction, the processing unit101 of the optimizing device 94 shown in FIG. 20 can be applied to autodriving following an arbitrary course.

[0459] First, the correction processing which the processing unit 101 ofthe optimizing device 94 shown in FIG. 20 performs will be describedwith reference to the flow chart in FIG. 22.

[0460] With the correction processing, in step S151, the correctionamount calculating unit 141 calculates the correction amount using thecorrection parameters a₀ through a_(N) stored in the correctionparameter memory 146.

[0461] That is, here, the correction amount is calculated with the truedriving direction θ being expressed as shown in Expression (37) usingthe correction parameters a₀ through a_(N) and the driving direction θ′obtained from the computing unit 93 as input signals, for example.

[0462] [Expression 37]

θ=θ′+a ₀ +a ₁ t ¹ +a ₂ t ² + . . . +a _(N) t ^(N)  (37)

[0463] Accordingly, a₀+a₁t¹+a₂t²+ . . . +a_(N)t^(N) is calculated at thecorrection amount calculating unit 141 as the correction amount, fromthe Expression (37). This correction amount is supplied to the computingunit 142.

[0464] In step S152, the driving direction θ′ obtained from thecomputing unit 53 and the correction amount are added at the computingdevice 142, and the added value thereof (θ in Expression (37)) is outputas output signals, the flow waits for a sample of the next input signalsto be supplied and returns to step S151, and subsequently, the sameprocessing is repeated.

[0465] Next, the correction parameter learning processing which theprocessing unit 101 of the optimizing device 94 in FIG. 20 performs willbe described with reference to the flowchart in FIG. 23.

[0466] With the correction parameter learning processing, first, in stepS161, the operating signal processing unit 110 judges whether or notlearning operating signals had been received from the operating unit 98(FIG. 16), and in the event that judgment is made that there has been noreception thereof, the flow returns to step S161.

[0467] Also, in the event that judgment is made in step S161 thatlearning operating signals have been received from the operating unit98, i.e., in the event of that, for example, the operating unit 98starts to be operated, following which, without elapsing of a first timet1 or longer, operating is performed continuously for a second time t2or longer, following which the operations thereof are stoppedcontinuously for a third time t3 or longer, or cases wherein followingstarting of operations of the operating unit 98, operations thereof arestopped continuously for a third time t3 or longer, such that judgmentcan be made that the user has operated the operating unit 98 so as todirect the automobile in the desired direction, the flow proceeds tostep S162, where the tutor data generating unit 143 generates tutordata, and the student data generating unit 144 generates student data.

[0468] That is to say, in the event of receiving a learning operatingsignal, the operating signal processing unit 110 supplies a learningmessage to that effect to the tutor data generating unit 143 and studentdata generating unit 144. Upon receiving the learning message, the tutordata generating unit 143 obtains the driving direction as input signalssupplied immediately thereafter as tutor data, and supplies this to thelearning data memory 53.

[0469] That is to say, in this case, for the tutor data, there is theneed to use the driving direction following the user operating theoperating unit 98 which is the steering wheel so that the automobileheads in the desired direction. Accordingly, the tutor data generatingunit 143 supplies the driving direction θ as input signals supplied thefollowing reception of the learning message, as tutor data, to thelearning data memory 53.

[0470] Also, the student data generating unit 52, upon receiving thelearning message, supplies the driving direction as input signalssupplied immediately prior to this, i.e., the driving directionimmediately before the automobile heads in the desired direction, asstudent data, to the learning data memory 53.

[0471] Following this, the flow proceeds to step S163, where thelearning data memory 53 stores the set of tutor data from the tutor datagenerating unit 51 and the student data, and the flow proceeds to stepS164.

[0472] In step S164, the same least-square adding as that described withExpressions (22) through (30) is performed by the correction parametercalculating unit 145, with regard to the tutor data and student data.

[0473] Note that the adding-in in step S164 is performed using theprevious adding results stored in the learning information memory 55 aslearning information, in the same manner as that described above. Also,here, adding is performed for obtaining, as θ′ in Expression (37), thecorrection parameters a₀ through a_(N) which minimize the sum of thesquare-error of the prediction value of the tutor data as θ inExpression (37) calculated using student data and the correspondingtutor data.

[0474] Following the adding in step S164, the correction parametercalculating unit 145 stores the adding-in results as learninginformation, in the form of overwriting of the learning informationmemory 55, and the flow proceeds to step S165.

[0475] In step S165, judgment is made regarding whether or not thecorrection parameter calculating unit 145 can obtain the correctionparameters a₁ through a_(N) from the adding results serving as learninginformation stored in the learning information memory 55.

[0476] In the event that judgment is made in step S165 that thecorrection parameters a₁ through a_(N) cannot be obtained, thecorrection parameters calculating unit 145 supplies information to thateffect to the judgment control unit 56, and the flow proceeds to stepS169. In step S169, the judgment control unit 56 supplies and storesdisable data representing forbidding of correction, as correctionparameters, to the correction parameter memory 146. The flow thenreturns to step S161, and subsequently, the same processing is repeated.

[0477] Accordingly, in the event that sufficient learning informationfor obtaining the correction parameters a₁ through a_(N) does not exist,correction of input signals is not performed at the correcting unit 21.That is, the correction amount of input signals is 0.

[0478] On the other hand, in the event that in step S165, judgment ismade that correction parameters, can be obtained, the flow proceeds tostep S166, where the correction parameter calculating unit 145 uses thelearning information to obtain the correction parameters a₀ througha_(N), supplies these to the judgment control unit 56, and the flowproceeds to step S167.

[0479] In step S167, the judgment control unit 56 obtains the predictionvalue of the corresponding tutor data from each of the student datastored in the learning data memory 53, following the Expression (37)defined by the parameter control data a₁ through a_(N) from thecorrection parameters calculating unit 145, and obtains the square sumof the prediction error of that prediction value (the error as to thetutor data stored in the learning data memory 53). Further, the judgmentcontrol unit 56 obtains a normalization error wherein the square sum ofthe prediction error thereof is divided by the number of learning pairsstored in the learning data memory 53, for example, and the flowproceeds to step S168.

[0480] In step S168, the judgment control unit 56 judges whether or notthe normalization error is greater than (equal to or more than) apredetermined threshold value S1. In the event that the judgment is madein step S168 that the normalization error is greater than thepredetermined threshold value S1, i.e., in the event that the linearexpression of the Expression (37) defined by the correction parametersa₁ through a_(N) does not precisely approximate the relation betweenstudent data and tutor data stored in the learning data memory 53, theflow proceeds to step S169, where the judgment control unit 56 suppliesand stores disable data to the correction parameter memory 146 ascorrection parameters, as described above. The flow then returns to stepS161, and subsequently, the same processing is repeated.

[0481] Accordingly, even in the event that the correction parameters a₁through a_(N) can be obtained, in the event that the Expression (37)defined by the correction parameters a₁ through a_(N) does not preciselyapproximate the relation between the student data and tutor data storedin the learning data memory 53, the correction amount of the inputsignals x(t) is 0, in the same way as cases wherein sufficient learninginformation for obtaining the correction parameters a₁ through a_(N)does not exist.

[0482] On the other hand, in the event that judgment is made in stepS168 that the normalization error is not greater than the predeterminedthreshold value S1, i.e., in the event that the linear expression of theExpression (37) defined by the correction parameters a₁ through a_(N)precisely approximates the relation between student data and tutor datastored in the learning data memory 53, the flow proceeds to step S170,where the judgment control unit 56 obtains the error ε between a planeof the Expression (37) defined by the correction parameters a₀ througha_(N) from the correction parameter calculating unit 145, and the pointstipulated by the newest tutor data and student data stored in thelearning data memory 53.

[0483] The flow then proceeds to step S171, where the judgment controlunit 56 judges whether or not the magnitude of the error ε is greaterthan (equal to or more than) a predetermined threshold value S2, and inthe event that judgment is made that this is not greater, step S172 isskipped, the flow proceeds to step S173, where the judgment control unit56 outputs the correction parameters a₁ through a_(N) obtained in stepS166 to the correction parameters memory 146. In this case, thecorrection parameters memory 146 stores the correction parameters a₀through a_(N) from the judgment control unit 56 in the form ofoverwriting, and the flow returns to step S161.

[0484] On the other hand, in the event that judgment is made in stepS171 that the magnitude of the error ε is greater than the predeterminedthreshold value S2, the flow proceeds to step S172, where the judgmentcontrol unit 56 controls the correction parameter calculating unit 145to re-calculate the correction parameters a₀ through a_(N), using onlythe newest tutor data and student data stored in the learning datamemory 53. The flow then proceeds to step S173, where the judgmentcontrol unit 56 outputs the correction parameters a₀ through a_(N)obtained in step S172 to the parameter control unit data memory 37, tostore in the form of overwriting, and the flow returns to step S161.

[0485] That is, in the same way as the embodiment in FIG. 8, with theembodiment in FIG. 23 as well, in step S170, the error ε is obtainedbetween a plane defined by the Expression (37) from the correctionparameters a₀ through a_(N) obtained from the tutor data and studentdata provided so far, and the point stipulated by the newest tutor dataand student data.

[0486] In the event that the magnitude of the error ε is not greaterthan the threshold value S2, the plane of the Expression (37) defined bythe correction parameters a₀ through a_(N) obtained in step S166 isconsidered to relatively precisely approximate the points stipulated bytutor data and student data provided so far, including the pointstipulated by the newest tutor data and student data, so the correctionparameters a₀ through a_(N) are stored in the correction parametermemory 146.

[0487] On the other hand, in the event that the magnitude of the error εis greater than the threshold value S2, the point stipulated by thenewest tutor data and student data is considered to relatively greatlydeviate from the plane of the Expression (37) defined by the correctionparameters a₀ through a_(N) obtained in step S166, so in step S172, thejudgment control unit 56 re-calculates the correction parameters a₀through a_(N) using only the newest tutor data and student data storedin the learning data memory 53.

[0488] Accordingly, in this case as well, learning is performed for thecorrection parameters a₀ through a_(N) of the Expression (37) based onlearning operating signals supplied according to operations made by theuser, so operations of the user can be learned without the user knowing,and further, processing which is optimal for the user can be performedusing the learning results.

[0489] Further, in this case, in the event that the error contained inthe running direction output by the computing unit 93 (FIG. 16) iscolored, the automobile can be made to perform auto-driving followingthe predetermined set course.

[0490] Also, with the processing unit 101 of the optimizing device 94 inFIG. 20, the correction parameters used in the correction processing(FIG. 6) performed at the correcting unit 21 are changed so that theactual driving direction of the automobile follows the set course,following the operations of the user made at the operating unit 98. Thatis, upon the user operating the operating unit 98 so that the drivingdirection of the automobile heads in the desired direction, learning isperformed of correction parameters, with each of the driving directionsθ immediately before and immediately after operating of the operatingunit 98 output by the computing unit 43 (FIG. 16) as input signals, asstudent data and tutor data, respectively, thereby changing thecorrection parameters. The changed correction parameters are supplied tothe correcting unit 21, the correction amount is calculated at thecorrecting unit 21 using the correction parameters, and correctionprocessing (FIG. 22) is performed for the input signals depending on thecorrection amount. Accordingly, in the event that the user operates theoperating unit 98, the correction parameters of the Expression (37) arechanged by the operations of the user, so that the contents of theprocessing (correction processing) represented by Expression (37) alsochange as a matter of course, so it can be said that the “contents ofprocessing” are being changed so that the driving direction desired bythe user can be obtained, following operations of the user, with theprocessing unit 101 of the optimizing device 94 shown in FIG. 20, aswell.

[0491] Further, with the optimizing device 94 in FIG. 20, in the eventthat sufficient learning pairs are not input from the user, or in theevent that learning pairs capable of highly precise approximation arenot input, the correction amount of input signals at the correcting unit21 is made to be 0, and in the event that learning pairs capable ofhighly precise approximation are input from the user, correction of theinput signals is performed by a correction amount obtained fromcorrection parameters obtained by learning being performed using thelearning pairs. That is, the algorithms for calculating weighting usedfor correction processing by the correcting unit 21 change between caseswherein sufficient numbers of learning pairs or learning pairs capableof precise approximation are not obtained, and cases wherein learningpairs capable of highly precise approximation are obtained.

[0492] Accordingly, with the processing unit 101 of the optimizingdevice 94 in FIG. 20, as well, the “contents of processing”, andfurther, the “structure of processing”, is also changed following useroperations, whereby the automobile performs auto-driving in the drivingdirection following the set course.

[0493] Now, with the embodiment in FIG. 23 (as with the embodiments inFIG. 8 and FIG. 13), in step S170, the error ε between the plane of theExpression (37) defined by the correction parameters a₀ through a_(N)from the correction parameters calculating unit 145, and the pointstipulated by the newest tutor data and student data, is obtained, andsubsequent processing is performed, but an arrangement may be madewherein, in step S170, the error ε between the plane of the Expression(37) defined by the correction parameters a₀ through a_(N) obtained instep S166 before multiple sets of recent tutor data and student data aresupplied, and each of points stipulated by multiple sets of recent tutordata and student data, is obtained, and subsequent processing isperformed based on the multiple errors ε.

[0494] Note that the processing unit 101 of the optimizing device 94 inFIG. 16 may be configured to use the processing unit 11 of theoptimizing device 1 shown in FIG. 10, for example, besides those shownin FIG. 17 and FIG. 20.

[0495] Next, FIG. 24 illustrates a configuration example of anotherembodiment of an auto-driving device to which he optimizing device shownin FIG. 1 has been applied. Note that in the figure, the partscorresponding to those in FIG. 16 are denoted by the same referencenumerals, and description thereof will be omitted where appropriate inthe following. That is to say, the auto-driving device shown in FIG. 24is configured in the same way as that shown in FIG. 16, other thanhaving an internal information generating unit 161 newly provided to inthe optimizing device 94, and further having a display device 171 newlyprovided.

[0496] In the same way as the interval information generating unit 171in FIG. 14, the internal information generating unit 161 reads out theinternal information from the processing unit 1, converts this intoimage information, and outputs to the display unit 171. The display unit171 displays the interval information supplied from the internalinformation generating unit 161 in a predetermined display format.

[0497] In the embodiment shown in FIG. 24, the processing unit 101 canbe configured as shown in FIG. 17 are FIG. 20. In the event that theprocessing unit 101 shown in FIG. 24 is configured in the same way asthat shown in FIG. 17, the same processing as that in FIG. 17 isperformed except for the correction parameter computing processing. Now,the correction parameter computing processing which the processing unit101 shown in FIG. 24 performs when configured as shown in FIG. 17, willbe described with reference to the flowchart in FIG. 25.

[0498] With steps S191 through S197, the same processing as that in eachof steps S111 through S117 in FIG. 18 is performed.

[0499] Following the processing in step S197, the flow proceeds to stepS198, and interval information is displayed to the display unit 171.That is, more specifically, for example, the internal informationgenerating unit 161 reads out the weight W stored in the weightingmemory 31 (FIG. 17) as internal information, converts this into imagesignals which can be displayed on the display unit 171, and outputs anddisplays (presents) on the display unit 171. Following the processingstep S198, the flow returns to step Sill, and subsequently, the sameprocessing is repeated.

[0500] Due to the processing described with the flowchart in FIG. 25,the weight W which is internal information regarding the processing ofthe processing unit 101 is displayed (presented) on the display unit171, and consequently, the user can operate the operating unit 98 whilewatching the display, so that optimal auto-driving is performed.

[0501] Also, in the above-described case, the weight W is displayed, butother internal information may be displayed (presented) on the displaydevice 171 with this internal information generating unit 161, forexample, parameter control data a and b can be read out from theparameter control data memory 37 and displayed. Also, an arrangement maybe made wherein the weight selected by the selecting unit 41 is weightobtained from the parameter control data a and b obtained by learningbeing performed using learning pairs, or weight obtained from the inputreliability or output reliability.

[0502] Next, in the event that the processing unit 101 of FIG. 24 isconfigured as shown in FIG. 20, processing the same as that performed inFIG. 20 is performed except for the correction parameter learningprocessing. Now, correction parameter learning processing in the casethat the processing unit 101 shown in FIG. 24 is configured as shown inFIG. 20, will be described, with reference to the flowchart shown inFIG. 26.

[0503] In steps S211 to S223, processing the same as that in each ofsteps S161 through S172 in FIG. 23 is performed.

[0504] Following the processing in step S219 and step S223, the flowproceeds to step S224, where, for example, the internal informationgenerating unit 161 reads out the correction parameters a₀ through a_(N)stored in the correction parameter memory as internal information,converts this into image signals that can be displayed on the displayunit 171, and displays on the display unit 171. At this time, thecorrection parameters a₀ through a_(N) configured of multipleparameters, another arrangement may be made wherein a display is made inthe form of a bar graph, for example, with parameters as the horizontalaxis and the values as the vertical axis, as shown in FIG. 27. Also, asshown in FIG. 28, for example, the correction parameters a₀ througha_(N) may be displayed with two arbitrary correction parameters a_(i)and a_(j) as the horizontal axis and vertical axis, respectively.Further, the arrangement may be made wherein the correction parametersto use as the horizontal axis and vertical axis may be selected by theuser.

[0505] Subsequently, the flow returns to step S211, and subsequently,the same processing is repeated.

[0506] As described above, due to the correction parameter processingdescribed with reference to the flowchart in FIG. 26, the correctionparameters a₀ through a_(N) are displayed, whereby the correctionparameters a₀ through a_(N) are displayed as internal information withthe processing unit 101 of the optimizing device 94 shown in FIG. 24, soconsequently, the user can operate the operating unit 98 while watchingthe display, so that optimal auto-driving is performed.

[0507] Further, with the internal information generating device 161,internal information other than the correction parameters a₀ througha_(N) may be displayed.

[0508] Also, in the embodiment shown in FIG. 26, in the event that theflow proceeds to step S224 following the processing in step S219, thecorrection parameters a₀ through a_(N) serving as the internalinformation are displayed as being 0.

[0509] Next, an optimizing device 201 as another embodiment of theoptimizing device shown in FIG. 1 will be described with reference toFIG. 29. The optimizing device 201 is configured of a processing unit211, and is for removing noise from image signals and the like, forexample serving as input signals, and optimizing displayed imagesignals. Now, in this example, image signals will be described as anexample of as primary input signals, the signals are not restricted toimage signals, and may be other signals as well.

[0510] The processing unit 211 is configured of a learning unit 221 anda mapping processing unit 222. Operating signals are supplied to thelearning unit 221 of the processing unit 211 from the operating unit202, and the learning unit 221 learns coefficient sets necessary for theprocessing with the mapping processing unit 222 based on the operatingsignals, and stores these in the coefficient memory 235. As for the normof learning of learning unit 211 (learning norm), for example, theleast-N′th-power error method (least-N′th-power method) may be used. Theleast-N′th-power error method will be described later.

[0511] The mapping processing unit 222 performs mapping processing formapping (converting) the input signals to predetermined output signals.That is, the mapping processing unit 222 takes, of the image signalsserving as output signals, a pixel yet to be obtained as a pixel ofinterest, extracts that tap (at least one or more pixel necessary forprocessing, also referred to as a sample) corresponding to the pixel ofinterest from the image signals which are input signals, and executesthe sum of products computing processing of coefficient sets stored inthe coefficient memory 235, thereby obtaining the pixel of interest. Themapping processing unit 222 performs processing (mapping) the same aswith pixels making up the image signals serving as output signals,generates image signals serving as output signals thereby which areoutput to the display unit 203, and displayed.

[0512] The operating unit 202 is operated by the user, and suppliesoperating signals corresponding to the operations, to the learning unit221. The display unit 203 displays pixel signals as output signalsoutput by the mapping processing unit 202.

[0513] Next, a detailed configuration of the learning unit 221 shown inFIG. 29 will be described, with reference to FIG. 30. The tutor datagenerating unit 231 generates tutor data serving as a tutor forlearning, from input signals, and outputs this to a least-N′th-powererror coefficient computing unit 234. The student data generating unit232 generates student data serving as a student for learning from theinput signals, and outputs this to a prediction tap extracting unit 233.This tutor data and student data is generated, for example, by the tutordata 231 performing no processing with the input signals, or by thestudent data generating unit 232 causing deterioration of input data bya predetermined thinning-out processing or LPF (Low Pass Filter) or thelike, etc., and is not restricted to the above-described configurationas long as student data is generated as that which is deteriorated withregard to the tutor data. Accordingly, besides that described above, forexample, in the event that the tutor data generating unit 231 subjectsthe input signals to predetermined thinning-out or LPF processing, it issufficient for the student data generating unit 232 to performthinning-out or LPF processing to a degree greater than that performedby the tutor data generating unit 231. Also, an arrangement may be madewherein the input signals are used without change as tutor data, andinput signals with noise superimposed thereupon is used as student data.

[0514] The prediction tap extracting unit 233 then sequentially takespixels configuring the image signals serving as tutor data as pixels ofinterest, extracts at least one or more pixels (taps) in a predeterminedpositional relation with pixel of interest as a prediction tap fromimage signals serving as student data, and outputs this to theleast-N′th-power error coefficient computing unit 234.

[0515] The least-N′th-power error coefficient computing unit 234computes coefficient sets from the prediction tap and tutor data byleast-N′th-power error, based on the operating signals representinginformation specifying the value of the exponent N necessary for theleast-N′th-power error coefficient computing processing input from theoperating unit 202, outputs this to the coefficient memory 235, andstores (performs overwriting storing as suitable).

[0516] The coefficient memory 235 stores the coefficient sets suppliedfrom the least-N′th-power error coefficient computing unit 234, andoutputs this to the mapping processing unit 222 where appropriate.

[0517] Next, the configuration of the mapping processing unit 222 inFIG. 29 will be described with reference to FIG. 31. The tap extractingunit 251 of the mapping processing unit 222 configures a prediction tapwith the same tap configuration as that configured by the prediction tapextracting unit 233 shown in FIG. 30 by sequentially extracting pixelsconfiguring image signals serving as output signals as pixels ofinterest from image signals serving as input signals, as prediction tapsof (the pixel value of) pixels in a predetermined positional relationwith pixels of interest, and outputs this to the sum of productscomputing unit 252. The sum of products computing unit 252 generatespixels of interest by performing the sum of products computation betweenthe values of extracted prediction taps (pixels) input from the tapextracting unit 251, and coefficient sets stored in the coefficientmemory 235 of the learning unit 221, and outputs this to the displayunit 203 (FIG. 29).

[0518] Now, coefficient computation by least-N′th-power error performedby the least-N′th-power error coefficient computing unit 234 shown inFIG. 30 will be described. A case wherein the exponent N=2 in theleast-N′th-power error method is commonly referred to as theleast-square error method (least-square-power error method). That is tosay, in a case wherein tutor data serving as a pixel of interest isrepresented by y, an M number of pieces of student data configuring aprediction tap by xi (i=1, 2, . . . , M), and a predetermined M numberof coefficients as w_(i), respectively, and the prediction value y′ ofthe tutor data y is a linear combination (sum of products computation)w₁x₁+w₂x₂+ . . . +w_(M)x_(M) of a prediction tap x_(i) and apredetermined coefficient w_(i), a coefficient set w₁, w₂, . . . , w_(M)which minimizes, as shown in FIG. 32, the sum of the square of the errorbetween the tutor data y indicated by black dots and the predictionvalues y′ indicated by white dots (in the absolute value of differencebetween the true value y serving as tutor data, and the prediction errory′ thereof, indicated by arrows in the figure), is obtained.

[0519] In the event of changing the value of the exponent N inleast-N′th-power error, the error of a prediction value y′ with greaterror greatly affects the sum of the N′th-power error in the event thatthe exponent N is great, for example, so consequently, with theleast-N′th-power error method, coefficients in the direction of wherebya such a prediction value y′ with a great error is saved (coefficientswhereby the error of the prediction value y′ with a great error isreduced) is obtained. However, the error of a prediction value y′ withlittle error has little effect on the sum of the N′th-power, and is notgiven much consideration, and consequently tends to be ignored.Conversely, in the event that the exponent N is small, the error of aprediction value y′ with great error affects the sum of the N′th-powerless than cases wherein the exponent N is great, and the error of aprediction value y′ with little error affects the sum of the N′th-powereven less. Consequently, according to the least-N′th-power error method,coefficients in the direction of reducing the error of the predictionvalue with small error is obtained in comparison with cases wherein theexponent N is great.

[0520] Note that the change in effect which the error of the predictionvalue y′ has on the sum of the N′th-power error is due to the error ofthe prediction value y′ to the N′th power, which may hereafter bereferred to as the nature of N′th-power error.

[0521] As described above, there is a qualitative tendency ofcoefficient sets obtained with least-N′th-power from the exponent N, socoefficient sets for executing mapping processing suiting thepreferences of the user can be obtained by changing the exponent N toobtain coefficient sets with the least-N′th-power method (coefficientsets which makes the output signals mapped with regard to the inputsignals, to be those preferred by the user). However, in reality, it isextremely difficult to compute coefficient sets which minimize the sumof the N′th-power error of a prediction value y′, with the method otherthan least-N′th-power error, i.e., a method wherein the exponent isother than N=2.

[0522] Now, the reason why computing coefficient sets which minimize thesum of the N′th-power error of the prediction value y′ with methodsother than least-square-power error will be described.

[0523] The sum of the error of the prediction value y′ to the N′th-power(the sum of the N′th-power error) can be represented by Expression (38).$\begin{matrix}{\left\lbrack {{Expression}\quad 38} \right\rbrack {E = {\sum\limits_{sample}^{\quad}^{N}}}} & (38)\end{matrix}$

[0524] Here, E indicates the sum of the number of samples of the N′thpower of the error e between the true value y serving as tutor data andthe prediction value y′.

[0525] On the other hand, with the present embodiment, as describedabove, the prediction value y′ of the true value y is defined by alinear combination of the prediction tap x_(i) and a predeterminedcoefficient wi, i.e., as with the following Expression (39).$\begin{matrix}{\left\lbrack {{Expression}\quad 39} \right\rbrack {Y^{\prime} = {{{w_{1}x_{1}} + {w_{2}x_{2}} + \ldots + {w_{M}x_{M}}} = {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}}}} & (39)\end{matrix}$

[0526] Now, the coefficients w₁, w₂, w₃, . . . , w_(M), may hereafter bereferred to as prediction coefficients. A set of the predictioncoefficients is the coefficients set stored in the coefficient memory235 shown in FIG. 30. Also, for the prediction taps x₁, x₂, x₃, . . . ,x_(M), pixels close space-wise or time-wise to the position of the imageserving as student data corresponding to the pixel (true value) y of theimage serving as tutor data, can be employed.

[0527] In this case, the error e in Expression (38) can be representedby the following Expression (40). $\begin{matrix}{\left\lbrack {{Expression}\quad 40} \right\rbrack {e = {\left| {y - y^{\prime}} \right| = \left| {y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}} \right|}}} & (40)\end{matrix}$

[0528] With the least-N′th-power error method, there is the need toobtain the prediction coefficients w₁, w₂, w₃, . . . , w_(M) whichminimize the sum E of the N′th-power error indicated in the followingExpression (41) and Expression (42) yielded from Expression (40). Notethat Expression (41) is an expression indicating the sum E in the eventthat the exponent N is an odd number, and Expression (42) is anexpression indicating the sum E in the event that the exponent N is aneven number. $\begin{matrix}{\left\lbrack {{Expression}\quad 41} \right\rbrack \quad {E = {{\sum\limits_{sample}^{\quad}^{N}} = {\sum\limits_{sample}^{\quad}\left| {y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}} \right|^{N}}}}} & (41) \\{\left\lbrack {{Expression}\quad 42} \right\rbrack \quad E = {{\sum\limits_{sample}^{\quad}^{N}} = {{\sum\limits_{sample}^{\quad}\left| {y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}} \right|^{N}} = {\sum\limits_{sample}^{\quad}\left( {y - {\sum\limits_{i = 1}^{M}{w_{i}x_{i}}}} \right)^{N}}}}} & (42)\end{matrix}$

[0529] Now, in the case of Expression (41), that is, in the event thatthe exponent N is an odd number, the sum E is the same value regardlessof the notation of the difference y−y′ if the sum of the magnitude ofthe difference y−y′ between the true value y and the prediction value y′is the same value, and consequently, the prediction coefficients w₁, w₂,w₃, . . . , w_(M) which minimize the sum E cannot be obtained. That isto say, the sum E is a function containing absolute values, and is, forexample a function such as shown in FIG. 33. Accordingly, the predictioncoefficients w₁, w₂, w₃, . . . , w_(M) which minimize the sum E cannotbe obtained except by a full search. FIG. 33 represents the change of aprediction coefficient w_(i) and the sum E in the event that theexponent N=1.

[0530] On the other hand, in the case of Expression (42), i.e., in theevent that the exponent N is an even number, the sum E

in the following Expression (43), the minimal value can be obtained bysetting an expression wherein the sum E in Expression (42) subjected topartial differentiation by the prediction coefficients w_(i) is 0.$\begin{matrix}{\left\lbrack {{Expression}\quad 43} \right\rbrack \begin{matrix}{\frac{\partial E}{\partial w_{i}} = {\sum\limits_{sample}^{\quad}{{N\left( {y - {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}}} \right)}^{N - 1} \cdot \left( {- x_{i}} \right)}}} \\{= {{- N}{\sum\limits_{sample}^{\quad}{x_{i}\left( {y - {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}}} \right)}^{N - 1}}}} \\{= 0}\end{matrix}} & (43)\end{matrix}$

[0531] Accordingly, from Expression (43), the solution, i.e., theprediction coefficients w₁, w₂, w₃, . . . , w_(M) which minimize the sumE of the N′th-power error, can be obtained by solving the expressionindicated in the following Expression (44). $\begin{matrix}{\left\lbrack {{Expression}\quad 44} \right\rbrack {{{\sum\limits_{sample}^{\quad}{x_{i}\left( {y - {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}}} \right)}^{N - 1}} = 0},\left( {{i = 1},2,\ldots \quad,M} \right)}} & (44)\end{matrix}$

[0532] With regard to this Expression (44), in the event that theexponent N−2 for example, i.e., in the event of obtaining the solutionwith the so-called least-square error method, substituting 2 for theexponent N in Expression (44) and solving the following Expression (45)is sufficient. $\begin{matrix}{\left\lbrack {{Expression}\quad 45} \right\rbrack {{{\sum\limits_{j = 1}^{M}{\sum\limits_{sample}^{\quad}{x_{i}x_{j}w_{j}}}} = {\sum\limits_{sample}^{\quad}{x_{i}y}}},\left( {{i = 1},2,\ldots \quad,M} \right)}} & (45)\end{matrix}$

[0533] As shown in the following Expression (46), the expression inExpression (45) can be represented in the form of a matrix expression,and is called a normal equation. Also, in the event that the exponentN=2, the minimal value of the sum E is uniquely determined, and thesmallest value thereof is the minimal value of the sum E. If a system oflinear equations of a number equal to the number of the predictioncoefficients w₁, w₂, w₃, . . . , w_(M) (M in this case) is configuredfrom the normal expression in Expression (45), the system of linearequations can be solved by the Cholesky method or the like, for example,and the prediction coefficients w₁, w₂, w₃, . . . , w_(M) can beobtained. $\begin{matrix}{\left\lbrack {{Expression}\quad 46} \right\rbrack \quad {{\begin{pmatrix}{\sum_{sample}{x_{1}x_{1}}} & {\sum_{sample}{x_{1}x_{2}}} & \cdots & {\sum_{sample}{x_{1}x_{M}}} \\{\sum_{sample}{x_{2}x_{1}}} & {\sum_{sample}{x_{2}x_{2}}} & \cdots & {\sum_{sample}{x_{2}x_{M}}} \\\vdots & \vdots & \vdots & \vdots \\{\sum_{sample}{x_{M}x_{1}}} & {\sum_{sample}{x_{M}x_{2}}} & \cdots & {\sum_{sample}{x_{M}x_{M}}}\end{pmatrix}\begin{pmatrix}w_{1} \\w_{2} \\\vdots \\w_{M}\end{pmatrix}} = \begin{pmatrix}{\sum_{sample}{yx}_{1}} \\{\sum_{sample}{yx}_{2}} \\\vdots \\{\sum_{sample}{yx}_{M}}\end{pmatrix}}} & (46)\end{matrix}$

[0534] Also, in order to solve the normal equation in Expression (46),it is necessary that the matrix having the sum of the product ofprediction taps x_(i)x_(j) (Σx_(i)x_(j)) at the left-hand side of theExpression (46) as a component is regular.

[0535] In the event that the exponent N is an even number and also is 4or more, the Expression (42) can be expressed as shown in the followingExpression (47). $\begin{matrix}{\left\lbrack {{Expression}\quad 47} \right\rbrack \quad {{{\sum\limits_{r = 0}^{N - 1}{\sum\limits_{sample}^{\quad}{x_{i} \cdot \left( {- 1} \right)^{r} \cdot {{}_{N - 1}^{}{}_{}^{}} \cdot y^{N - 1 - r} \cdot \left( {\sum\limits_{j = 1}^{M}{w_{j}x_{j}}} \right)^{r}}}} = 0},\left( {{i = 1},2,\ldots \quad,M} \right)}} & (47)\end{matrix}$

[0536] With the expression shown in Expression (47), this forms a systemof equations of a high order, so the prediction coefficients w₁, w₂, w₃,. . . , w_(M) cannot be obtained by solving a system of linearexpressions as with the case wherein the exponent N is N=2.

[0537] As described above, in the event that the exponent N is otherthan N=2, the prediction coefficients w₁, w₂, w₃, . . . , w_(M) whichminimize the sum of N′th-power error indicated in Expression (38)generally cannot be readily obtained.

[0538] Accordingly, the least-N′th-power error coefficient computingunit 234 of the learning unit 221 computes the prediction coefficientsby the following two least-N′th-power error methods. Which of the twoleast-N′th-power error methods to employ can be specified by, forexample, the user operating the operating unit 202 (FIG. 29).

[0539] First, a first method (hereafter also referred to as the directtechnique) will be described. As indicated in the following Expression(48), the sum E wherein the item of the error e² has been multiplied bythe weight α_(s) is defined as the sum of the N′th-power error insteadof the Expression (38). $\begin{matrix}{\left\lbrack {{Expression}\quad 48} \right\rbrack {E = {{\sum\limits_{sample}^{\quad}^{N}} = {\sum\limits_{sample}^{\quad}{\alpha_{s}^{2}}}}}} & (48)\end{matrix}$

[0540] That is, the e^(N) which is error to the N′th power is defined bythe product of the weight α_(s) and the error squared e².

[0541] In this case, the prediction coefficients w₁, through w_(M) whichminimize the sum E of N′th-power error in Expression (48) can beobtained by making the weight α_(s) in Expression (48) to be a functionof the prediction value y′ obtained by the linear expression inExpression (39), from the prediction coefficients w₁, w₂, w₃, . . . ,w_(M) obtained in the event that the exponent N is N=2 as shown in thefollowing Expression (49), for example.

[0542] [Expression 49]

α_(s) =f(y′)  (49)

[0543] While various function can be conceived for the weight α_(s),there is the need to employ a function wherein the error to the N′thpower e^(N)=α_(s)e² defined in Expression (48) satisfies theabove-described nature of the N′th-power error, and the function shownin the following Expression (50), for example, can be employed.

[0544] [Expression 50]

α_(s) =ax _(s) ^(c) +b  (50)

[0545] Now, x_(s) represents a value wherein the error of the predictionvalue y′ computed by Expression (39) from the prediction coefficientsw₁, through w_(M) obtained by least-square error (hereafter may bereferred to as error from least-square norm) is normalized to a range of0 through 1.0, and the weight α_(s) in the Expression (50) defined withthe error x_(s) of the prediction value y′ is as shown in FIG. 34.

[0546] The coefficient a is an item for controlling the effect which theerror x_(s) from the least-square norm has on the N′th-power errore^(N), and in the event that the coefficient a is a=0, the weight α_(s)is a horizontal line with an inclination of 0 in FIG. 34. In this case,the effect which the error x_(s) from the least-square norm has on theN′th-power error e^(N) is constant regardless of how large or how smallthe error x_(s) from the least-square norm is, and the predictioncoefficient w_(i) which minimizes the sum E in Expression (48) logicallyis the same as that obtained by least-square error. Accordingly, makinga=0 essentially realizes the least-square error method. Also, in theevent that coefficient a>0 holds,

[0547] the effect which the error x_(s) from the least-square norm hason the N′th-power error e^(N) in Expression (48) is greater the greaterthe error x_(s) is, and is smaller the smaller the error x_(s) is. Onthe other hand, in the event that coefficient a<0 holds, the effectwhich the error x_(s) from the least-square norm has on the N′th-powererror e^(N) in Expression (48) is smaller the greater the error x_(s)is, and is greater the smaller the error x_(s) is.

[0548] That is to say, in the event that the coefficient a is positive,the N′th-power error e^(N) in Expression (48) has the same nature asmaking the exponent N to be large, and in the event that the coefficienta is negative, this has the same nature as making the exponent N to besmall. Accordingly, the N′th-power error e^(N) in Expression (48) hasthe same nature as the N′th-power error e^(N) in Expression (38), soprediction coefficients which minimize the sum E of the N′th power errorin Expression (48) essentially is the prediction coefficients whichminimize the sum E of the N′th power error in Expression (38).

[0549] Also, in the event that the coefficient a is 0, the least-squareerror is realized, as described above. That is, the exponent N is 2, butwith cases wherein the coefficient a is positive, this makes for caseswherein the exponent N is N>2, and with cases wherein the coefficient ais negative, this makes for cases wherein the exponent N is N<2. Thiscoefficient a greatly affects the exponent N in least-N′th-power error,as with the later-described coefficient c.

[0550] The coefficient b is a correction item, and the overall functionvalue (weight α_(s)) in FIG. 34 changes in the vertical directionaccording to the value of the coefficient b. Also, the coefficient bdoes not have that much of an effect on the exponent N withleast-N′th-power error.

[0551] The coefficient c is an item which converts the scaling of theaxes, i.e., an item which changes the way the weight α_(s) isappropriated to the error x_(s) from the least-square norm, wherein thegreater the value of the coefficient c is the more precipitous thechange in the weight α_(s) is, and the smaller the value of thecoefficient c is the gentler the change in the weight α_(s) is.Accordingly, the effect which the error x_(s) from the least-square normhas on the N′th-power error e^(N) in Expression (48) due to thecoefficient c changing is the same as that of the coefficient achanging, so the N′th-power error e^(n) in Expression (48) can beprovided with the same nature as the N′th-power error e^(N) inExpression (38) with the coefficient c, as well. That is to say, theexponent N of the least-N′th-power error can be affected by thecoefficient c.

[0552] Note that the line in FIG. 34 indicates a case wherein c=1 anda>0 (b is arbitrary), and the curve in FIG. 34 indicates a case whereinc≠1 and a>0 (b is arbitrary).

[0553] The coefficients a, b, and c, which stipulate the weight α_(s) inExpression (50) may be changed by the user operating (setting) theoperating unit 202, and the weight α_(s) in Expression (50) changes bychanging the coefficients a, b, and c. This change in the weight α_(s)causes the α_(s)e² in Expression (48) to essentially (equivalently)function as the error to the N′th power e^(N) with regard to apredetermined exponent N, and consequently, the prediction coefficientswhich minimize the sum E of N′th-power error in Expression (50), i.e.,the prediction coefficient w_(i) of the norm by least-N′th-power error,to be obtained.

[0554] Also, with the above-described first method, variously changingthe coefficients a, b, and c, which determine the weight α_(s)essentially changes the exponent N so as to obtain the predictioncoefficients by least-N′th-power error, so the exponent N is notrestricted to integers, and the prediction coefficients can be obtainedin the event that the exponent N is another real number such as adecimal, e.g., wherein the exponent N=2.2.

[0555] Next, a second method for computing prediction coefficients byleast-N′th-power error (hereafter also referred to as the recursivemethod) will be described. With the first method, the squared error e²multiplied by the weight α_(s) is employed as the N′th-power error asshown in Expression (48), but with the second method, a solutionobtained with lower order least-N′th-power error is used, and a solutionby higher order least-N′th-power error can be obtained with a recursivetechnique.

[0556] That is, the prediction coefficient w_(i) which minimizes the sumE of square-error in the following Expression (51) can be obtained withthe least-square error method as described above, with the predictionvalue y′ computed by the Expression (39) using the predictioncoefficient w_(i) obtained by this least-square error being representedby y₁ (hereafter may be referred to as prediction value fromleast-square norm).

[0557] [Expression 51]

E=Σe ²  (51)

[0558] Next, let us consider a sum E of a cubic error expressed in thefollowing Expression (52), for example.

[0559] [Expression 52]

E=Σ|e| ³=Σ(|y−y ₁)e ²  (52)

[0560] Obtaining the prediction coefficient w_(i) which minimizes thesum E of the cubic-error in Expression (52) means obtaining a solutionby least-cubic error, wherein the cubic-error |e³| is expressed in theform of the product of the square-error e² and the error |y−y₁| betweenthe prediction value y₁ from the least-square norm and the true value y,as shown in Expression (52). The |y−y₁| in Expression (52) can beobtained as a constant, so the prediction coefficient w_(i) whichminimizes the sum E of the cubic-error in Expression (52) can actuallybe obtained by least-square error.

[0561] In the same way, let us consider a sum E of a biquadrate errorexpressed in Expression (53).

[0562] [Expression 53]

E=Σe ⁴=Σ(|y−y₂|)² e ²  (53)

[0563] Obtaining the prediction coefficient w_(i) which minimizes thesum E of the biquadrate-error in Expression (53) means obtaining asolution by least-biquadrate error, wherein, with the prediction valuey′ computed by Expression (39) using the prediction coefficient w_(i)which minimizes the sum E of the cubic-error in Expression (529) beingrepresented by y₂ (hereafter may be referred to as prediction value fromleast-cubic norm), the biquadrate-error e⁴ can be expressed in the formof the product of the square-error e² and the squared error |y−y₂|²between the prediction value y₂ from the least-cubic norm and the truevalue y (hereafter may be referred to as square-error from least-cubicnorm), as shown in Expression (53). The squared error |y−y₂|² from theleast-cubic norm in Expression (53) can be obtained as a constant, sothe prediction coefficient w_(i) which minimizes the sum E of thebiquadrate-error in Expression (53) can actually be obtained byleast-square error.

[0564] This is also true for the following Expression (54).

[0565] [Expression 54]

E=Σ|e| ⁵=Σ(|y−y ₃|)³ e ²  (54)

[0566] Obtaining the prediction coefficient w_(i) which minimizes thesum E of the fifth-power-error in Expression (54) means obtaining asolution by least-fifth-power error. With the prediction value y′computed by Expression (39) using the prediction value w_(i) whichminimizes the sum E of the biquadrate-error in Expression (53) beingrepresented by y₃ (hereafter may be referred to as prediction value fromleast-biquadrate norm), the fifth-power-error e⁵ can be expressed in theform of the product of the square-error e² and the cubed error |y−y₃|³between the prediction value y₃ from the least-biquadrate norm and thetrue value y (hereafter may be referred to as cubic-error fromleast-biquadrate norm), as shown in Expression (54). The cubed error ofthe least-biquadrate norm in Expression (54) can be obtained as aconstant, so the prediction coefficient w_(i) which minimizes the sum Eof the fifth-power-error in Expression (54) also can actually beobtained by least-square error.

[0567] The solution (prediction coefficient w_(i)) can also be obtainedin the same manner with cases of least-N′th power error wherein theexponent N is 6 or a higher order.

[0568] As described above, with the second method, a prediction valuecomputed using a prediction coefficient obtained with lower orderleast-N′th-power error is used to obtain a solution of higher orderleast-N′th-power error, and this is recursively repeated, therebyobtaining a solution with higher order least-N′th-power error.

[0569] Note that with the above-described case, the solution fromleast-N′th-power error is obtained using a prediction value computedusing a prediction coefficient obtained by N-1′th-power error which islower thereby by just the order of 1, but the solution byleast-N′th-power error may be obtained using a prediction value computedusing a prediction coefficient obtained by least-N′th-power error of anarbitrary order lower than that. That is, in the case of Expression(53), |y−y₁| may be used instead of |y−y₂|, and in the case ofExpression (54), |y−y₂| or |y−y₁| may be used instead of |y−y₃|.

[0570] Also, in the second method, the N′th-power-error e^(N) can beexpressed in the form of the product of the square-error e² and theN−2′th-power error |y−y′|^(N−2), so as with the first method, a solutioncan be obtained by least-N′th error with an arbitrary exponent N, e.g.,wherein the exponent N=2.2.

[0571] Next, the image optimizing processing with the optimizing device201 in FIG. 29 will be described with reference to the flowchart in FIG.35. This image optimizing processing is made up of learning processingand mapping.

[0572] In the learning processing, in Step S230, the judgment is maderegarding whether or not the user has operated the operating unit 202,and in the event that judgment is made that this has not been operated,the flow returns to Step S230. Also, in Step S230, in the event thatjudgment is made the operating unit 202 has been operated, the flowproceeds to Step S231.

[0573] In Step S231, the tutor data generating unit 231 of learning unit221 generates tutor data from the input signals and outputs this to theleast-N′th-power error coefficient computing unit 234, while the studentdata generating unit 232 generates student data from the input signalsand outputs this to the prediction tap extracting unit 233, and the flowproceeds to Step S232.

[0574] Now, as for data used for generating the student data and tutordata (hereafter may be referred to as learning data), input signalsinput to a point back in time from the current time by a predeterminedamount of time may be employed, for example. Also, for the learningdata, an arrangement may be made wherein dedicated data is storedbeforehand, instead of using input signals.

[0575] In Step S232, with each piece of tutor data as pixel of interest,the prediction tap extracting unit 233 generates prediction tap from thestudent data input by the student data generating unit 232 with regardto each pixel of interest, and outputs this to the least-N′th-powererror coefficient computing unit 234, and the flow proceeds to StepS233.

[0576] In Step S233, the least-N′th-power error coefficient computingunit 234 judges whether or not operating signals specifying computationof a coefficient set by least-N′th-power error with the recursive method(second method) has been input by the operating unit 202, and in theevent that, for example, the operating unit 202 has been operated by theuser and a method which is not the recursive method has been specified,i.e., the direct method (first method), the flow proceeds to Step S234,where the judgment is made regarding whether or not coefficients a, b,and c, which specify the weight α_(s) in Expression (50) (specify theexponent N), the processing is repeated until there is input thereof,and in the event that judgment is made that, for example, the operatingunit 202 has been operated by the user and the values specifying thecoefficients a, b, and c, have been input, the processing proceeds toStep S235.

[0577] In Step S235, the least-N′th-power error coefficient computingunit 234 essentially solves the problem of minimizing theabove-described Expression (48) in the state of the coefficients a, b,and c with the weight α_(s) input by the least-square error method,thereby obtaining the prediction coefficients w₁, w₂, w₃, . . . , w_(M),has a solution with least-N′th-power error for the exponent Ncorresponding to the weight α_(s), i.e., a coefficient set, and storesthis in the coefficient memory 235, and the flow returns to Step S230.

[0578] On the other hand, in Step S233, in the event that judgment ismade that the recursive method has been selected, the processingproceeds to Step S236.

[0579] In Step S236, the least-N′th-power error coefficient computingunit 234 judges whether or not information specifying the exponent N hasbeen input, repeats the processing until the exponent N is input, and inthe event that judgment is made that information specifying the exponentN has been input by the user operating the operating unit 202, forexample, the processing proceeds to Step S237.

[0580] In Step S237, the least-N′th-power error coefficient computingunit 234 obtains the coefficient set by an algebraic approach using thebasic least-square error method. In Step S238, the least-N′th-powererror coefficient computing unit 234 uses a prediction value obtained bythe coefficient set obtained by least-square-error, obtains thecoefficient set by a least-N′th-power error input from the operatingunit 202 corresponding to the exponent N recursively, as described withreference to Expression (51) through Expression (54), which is stored inthe coefficient memory 235, and the flow returns to Step S231.

[0581] Next, with the mapping processing, in Step S241, with a frame ofan image serving as output signals corresponding to an image framesserving as current input signals, as a frame of interest, the tapextracting unit 251 of the mapping processing unit 222 takes, of pixelsof the frame of interest, those which have not yet been the pixel ofinterest, in raster scan order, for example, extracts a prediction tapfrom the input signals for that pixel of interest, and outputs this tothe sum of products computing unit 252.

[0582] Then, in Step S242, the sum of products computing unit 252 readsout prediction coefficients of the coefficient memory 235 of thelearning unit 221, and executes sum of products computing processingbetween the prediction tap input from the tap extracting unit 251 andthe prediction coefficient read out from the coefficient memory 235,following the Expression (39). Thus, the sum of products computing unit252 obtains the pixel value (prediction value) for the pixel ofinterest. Subsequently, the flow proceeds to Step S243, where the tapextracting unit 251 judges whether or not all pixels have been made thepixel of interest in the frame of interest, and in the event thatjudgment is made that there are some that have not, the flow returns toStep S241, and the same processing is repeated with pixels in the frameof interest which have not yet been made the pixel of interest, beingmade the new pixel of interest, in raster scan order.

[0583] Also, in Step S243, in the event that judgment is made that allpixels in the frame of interest have been made to be the pixel ofinterest, the flow proceeds to Step S244, where the display unit 203displays the frame of interest made up of the pixels obtained by the sumof products computing unit 252.

[0584] The flow returns to Step S241, where the tap extracting unit 251takes the next frame as a new frame of interest, and subsequently, thesame processing is repeated.

[0585] According to the image optimizing processing in FIG. 35, in themapping processing, the user views the image displayed on the displayunit 203 and in the event this does not match the preferences of theuser, operates the operating unit 202 to specify direct method orrecursive method, and further specify the exponent N forleast-N′th-power error, and accordingly in the learning processing, theprediction coefficients obtained by least-N′th-error are changed, andthe image which is the output signals obtained by the mapping processingcan be made to be that which suits the preferences of the userhim/herself.

[0586] Now, FIG. 36 illustrates the sum of error of prediction valuescomputed using a coefficient set of the least-N′th-power norm obtainedwith least-N′th-power error, and the sum of error of prediction valuescomputed using a coefficient set of a least-square norm obtained by ageneral least-square error method, with the direct method wherein thevalues of the coefficients a, b, and c, of the weight α_(s) inExpression (50) are changed, by setting the coefficients a, b, and c to,for example, a=40, b=0.1, and c=1. Here, the sum of the square error andcubic error is indicated for the sum of error. Also, in the event thatthe coefficients a, b, and c, are the above-described values, this isequivalent to a case wherein the exponent N of the N′th-power errore^(N) in Expression (48) is greater than 2. In FIG. 36, the sum ofsquare error is 10160281 in the case of coming from the least-squarenorm coefficient set, and is 10828594 in the case of coming from a leastN′th-power norm coefficient set, so the case of the least-square normcoefficient set is smaller than the case of the least N′th-power normcoefficient set. On the other hand, the sum of cubic error is 165988823in the case of coming from the least-square norm coefficient set, and is161283660 in the case of coming from a least N′th-power norm coefficientset, so the case of the least N′th-power norm coefficient set is smallerthan the case of the least-square norm coefficient set.

[0587] Accordingly, an image which is output signals wherein the sum ofthe squared norm is even smaller can be obtained by performing themapping processing (the sum of products computation in Expression (39))using the coefficient set of the least squared norm, and also, an imagewhich is output signals wherein the sum of the cubic norm is evensmaller can be obtained by performing the mapping processing using thecoefficient set of the least-N′th-power norm obtained using thecoefficients a, b, and c, of the above-described values.

[0588] Now, in the image optimizing processing in FIG. 35, the exponentN is changed by the user operating the operating unit 202 (in the directmethod, the coefficients a, b, and c, specifying the exponent N arechanged, and in the recursive method, the exponent N itself is changed)thus setting what sort of least-N′th-power error method for the exponentN is to be employed as the learning norm (learning system) forprediction coefficients (coefficient set). That is, the learningalgorithms for obtaining prediction coefficients themselves are changed.Accordingly, it can be said that the “structure of processing” is beingchanged so that an image suiting the preferences of the user can beobtained.

[0589] Next, FIG. 37 illustrates another configuration example of theoptimizing device shown in FIG. 1. With the optimizing device 201 shownin FIG. 37, configurations other than an internal information generatingunit 261 being provided are the same as those of the optimizing device201 shown in FIG. 29, so description thereof will be omitted.

[0590] The internal information generating unit 261 reads out, forexample, prediction coefficients stored in the coefficient memory 235 asinternal information of the processing unit 211, converts theinformation of the prediction coefficients into image signals andoutputs this to the display unit 203, where it is displayed.

[0591] Next, the image optimizing processing with optimizing device 201shown in FIG. 37 will be described with reference to the flowchart shownin FIG. 38. The image optimizing device shown in FIG. 38 is also made upof learning processing and mapping processing, as with the case shown inFIG. 35. With the learning processing, processing the same of that insteps S230 through S238 is performed in steps S250 through S258,respectively.

[0592] Further, with the learning processing, following the processingin steps S255 and S258, the flow proceeds to step S259, for the internalinformation generating unit 261 reads the coefficient set stored in thecoefficient memory 235 as internal information, generates image signalswhich can be displayed based on the values contained in the coefficientset and outputs this to the display unit 203 where it is displayed.

[0593] At this time, the image generated by the internal informationgenerating unit 261 and displayed on the display unit 203 may have aform such as a 3-dimensional distribution diagram such as shown in FIG.39, or a 2-dimensional distribution diagram such as shown in FIG. 40,for example. That is, in FIG. 39, coordinates equivalent to theprediction tap position extracted from the input signals are indicatedas positions on an x y plane, as Tap Position (x) and Tap Position (y),with prediction coefficients (Coeff) taking the product of pixel valuesserving as the prediction tap thereof shown at the coordinatescorresponding to each of the prediction tap positions. Also, in FIG. 40,FIG. 39 is represented in the form of contour lines.

[0594] Now, let us return to the description of the flowchart in FIG.38.

[0595] Following the processing in Step S259, the flow returns to stepS250, and subsequently, the same processing is repeated.

[0596] On the other hand, with the mapping processing, processing thesame of that in steps S241 through S244 in FIG. 35 is performed in stepsS261 through S264, respectively.

[0597] Due to the above processing, the values of the coefficient sets(the coefficient values) stored in the coefficient memory 235 of theprocessing unit 211 is displayed (present) as internal informationrelating to the processing, so the user operates the operating unit 202while viewing the distribution of the coefficient sets and theprocessing results of the processing unit 211 as output signals so thatan image as output signals suiting the preferences thereof can beobtained, thereby changing the exponent N (in the direct method, thecoefficients a, b₇ and c, specifying the exponent N are changed, and inthe recursive method, the exponent N itself is changed) thus settingwhat sort of least-N′th-power error method for the exponent N is to beemployed as the learning norm (learning system) for predictioncoefficients (coefficient set). That is, the learning algorithms forobtaining prediction coefficients themselves are changed, so it can besaid that the “structure of processing” is being changed. Also, in theabove-described example, the coefficient sets are displayed, butarrangement may be made wherein internal information relating toprocessing is displayed such as, for example, whether the currentleast-N′th-power error method is direct or recursive.

[0598]FIG. 41 illustrates another configuration example of theoptimizing device. The optimizing device 301 shown in FIG. 41 isconfigured of the processing unit 311, and optimizes input signals anddisplays these on a display unit 202 based on operating signals inputfrom the operating unit 202. In the figure, parts corresponding to theabove-described embodiments are denoted with the same referencenumerals, and description thereof will be omitted in the following,wherever appropriate.

[0599] The coefficient memory 321 of the processing unit 311 shown inFIG. 41 is basically the same as the coefficient memory 235 shown inFIG. 30, and stores coefficient sets necessary for executing the mappingprocessing by the mapping processing unit 222. The coefficient sets arebasically coefficient sets generated by the learning device 341 shown inFIG. 43 which will be described later (coefficient set serving asinitial values), which are changed by the coefficient changing unit 322as appropriate, and overwritten and stored. Accordingly, as overwritingis repeated, the coefficient sets come to be something different to thatgenerated by the learning device 341. Also, an arrangement may be madewherein the coefficient sets serving as the initial value are stored inunshown memory, so that the storage contents of the coefficient memory321 can be reverted to the initial value coefficient set, correspondingto operations made by the operating unit 202.

[0600] The coefficient changing unit 322 reads out coefficient setstored in the coefficient memory 321 (prediction coefficient), based onoperating signals input from the operating unit 202, changes the valuesof the prediction coefficients (prediction coefficient multiplied by theprediction taps) corresponding to the prediction taps, and overwrites onthe coefficient memory 321 again and stores.

[0601] Next, the configuration of the coefficient changing unit 322 willbe described with reference to FIG. 42. The coefficient read/write unit331 of the coefficient changing unit 322 is controlled by the changeprocessing unit 332, so as to read out the coefficient sets stored inthe coefficient memory 321, and output to the changing processing unit332, while writing the prediction coefficients, of which the values havebeen changed by the changing processing units 332, on the coefficientmemory 321 for storage. The changing processing unit 332 changes theprediction values read out from the coefficient memory 321 by thecoefficient read/write unit 331, based on the operating signals.

[0602] Here, a learning device 341 for generating coefficient sets to bestored in the coefficient memory 321 by the learning processing, will bedescribed with reference to FIG. 43. The tutor data generating unit 351of the learning device 341 is the same as the data generating unit 231of the learning device 221 shown in FIG. 30, and generates tutor datafrom image signals which is learning data provided beforehand, andoutputs tutor data to the normal equation generating unit 354. Thestudent data generating unit 352 is the same as the student datagenerating unit 321 shown in FIG. 30, and generates student data fromthe learning data and outputs this to the prediction tap extracting unit353.

[0603] The prediction tap extracting unit 353 is the same as theprediction tap extracting unit 233 shown in FIG. 307 wherein, with thatthe tutor data which is to be processed now as the pixel of interest,extracts a prediction tap of the same tap structure as that generated bythe tap extracting unit 251 (FIG. 31) making up the mapping processingunit 222 shown in FIG. 41 from the student data with regard to thatpixel of interest, and outputs this to the normal equation generatingunit 354.

[0604] The normal equation generating unit 354 generates a normalequation of the Expression (46) from the tutor data y which is the pixelof interest input from the tutor data generating unit 351 and predictiontaps x₁, x₂, . . . , x_(M). The normal equation generating unit 354 thenobtains the normal equation in Expression (46) with all tutor data aspixel of interest, and outputs the normal equation to the coefficientdeciding unit 355. The coefficient deciding 355 solves the input normalequation (the above-described Expression (46)) by the Cholesky method,or the like, thereby obtaining coefficient sets.

[0605] Next, coefficient deciding processing (learning processing) withthe learning device 341 shown in FIG. 43 will be described, withreference to the flowchart in FIG. 44. In step S271, the tutor datagenerating unit 351 generates tutor data from the learning data, andoutputs this to the normal equation generating unit 354, while thestudent data generating unit 352 generates student data-from thelearning data, outputs this to the prediction tap extracting unit 353,and the flow proceeds to step S272.

[0606] In step S272, the prediction tap extracting unit 352 sequentiallytakes tutor data as pixels of interest, extracts prediction taps fromthe student data with regard to the data of interest, outputs this tothe normal equation generating unit 354, and the flow proceeds to stepS273.

[0607] In step S273, the normal equation generating unit 354 uses thetutor data and prediction tap sets to compute the summation (Σ) of thecomponents of the matrix at the left-hand side of the Expression (46)and the summation (Σ) of the components of the vector of the right-handside thereof, thereby generating a normal equation, which is output tothe coefficient deciding unit 355.

[0608] The flow then proceeds to step S274, for the coefficient decidingunit 355 solves the normal equation input from the normal equationgenerating unit 354 and obtains a coefficient set by so-calledleast-square error, which is stored in the coefficient memory 321 instep S275.

[0609] Due to the above processing, a basic coefficient set (coefficientset serving as an initial value) is stored in the coefficient memory321. Now, while the above description gives the coefficient set beingobtained by least-square error, this may be a coefficient set obtainedby a another method, and may be a coefficient set obtained by theabove-described least-N′th-power method.

[0610] Next, description will be made regarding a changing of thecoefficient set by the changing processing unit 332 of the coefficientchanging unit 322 shown in FIG. 41. Coefficient sets are stored in thecoefficient memory 321 beforehand by the processing of the flowchart inthe above-described FIG. 44, and the coefficient changing unit 322changes the prediction coefficients of the coefficient set that is setby computation beforehand, based on operating signals input from theoperating unit 202.

[0611] For example, in the event that the prediction taps extracted fromthe student data are 7 taps×7 taps (7×7 pixels for horizontal×vertical),a total of 49 taps, the same number of prediction coefficientcorresponding to the prediction taps exist. That is, in this case, acoefficient set stored in the coefficient memory 321 is a coefficientset made up of 49 prediction coefficients. At this time, with theposition of each of the prediction taps (tap position) as the horizontalaxis (e.g., providing numbers to each of the prediction taps and usingthe value of the number as a horizontal axis), let us say that thedistribution, wherein the coefficient value of the predictioncoefficient multiplied by the prediction tap of the tap positions istaken as the vertical axis, is as shown in FIG. 46. Now, in the eventthat all of the coefficient values of the coefficient set are to bechanged, this requires operations for each of the 49 coefficient values.It is necessary that the values of the coefficient be normalized(wherein each of the coefficients are divided by the total of the valuesof all coefficient) so that gain of the input signals and the outputsignals obtained by processing the input signals with predictioncoefficient is the same, i.e., that the total value of the coefficientbe 1, but it is difficult to operate each of the individual coefficientso that the total is 1.

[0612] That is to say, for example, considering raising only thecoefficient value of the coefficient corresponding to the prediction tapindicated by the arrow in FIG. 47, the distribution of tap positions andcoefficient values is as shown in FIG. 48. Note that the tap position tshown in FIG. 48 is the tap position indicated by the arrow in FIG. 47.

[0613] In this way, raising (increasing) only the coefficient value ofthe coefficient corresponding to a particular prediction tap requiresoperations so that the coefficient values of coefficient correspondingto other tap positions are reduced or the like such that the total is 1,but the operations are difficult. Further, operations for making thetotal 1 is similarly difficult in cases of changing the coefficientvalue of coefficient corresponding to a greater number of predictiontaps, as well.

[0614] Accordingly, in the event that the operating signals are suchthat the coefficient value of a coefficient corresponding to one tapposition indicated by the arrow shown in FIG. 49 is to be changed beyonda predetermined threshold value S11 (the amount of change changesgreater than a threshold value S11), the change processing unit 332changes the coefficient value of other coefficient from the distributionsuch as shown in FIG. 46 to a distribution such as shown in FIG. 50.That is to say, the change processing unit 332 changes the coefficientvalues of coefficient corresponding to each of the tap positions, suchthat the distribution of coefficient values changes according to a modelsuch as like a spring, according to the distance from the tap positioncorresponding to the coefficient regarding which the value thereof hasbeen changed. That is, in the event that the distribution of thecoefficient set obtained by learning is a distribution such as shown inFIG. 46, upon operations being made at the operating unit 202 so as toraise the value corresponding to the tap position t as shown in FIG. 50,the change processing unit 332 increases the coefficient value for othervalues at positions close to the tap position t such that the closer theposition is, the greater the changes, and conversely, the coefficientvalues of coefficients corresponding to taps at positions distant fromthe tap position t are changed so that the farther the position, thegreater the decrease is, and further, where the total of coefficientvalues is 1. Here, in the following, the model wherein the distributionchanges in the form of a spring such as shown in FIG. 50 will be calleda spring model. Further, according to the spring model, in the eventthat operating unit 202 is operated so as to lower the coefficient of acertain tap position t, the coefficient values of positions close to thetap position t are lowered corresponding to the closeness to thatposition, and the coefficient values of positions distant from theposition t are conversely raised according to the distance from thatposition.

[0615] Also, in the event that the amount of change of the coefficientvalue of the coefficient is smaller than the predetermined thresholdvalue S11, the change processing unit 332 changes coefficient valueshaving a extreme value the same as the polarity of the coefficient ofthe tap position t in the same direction as the coefficient of the tapposition t, and changes coefficient values having a extreme valuedifferent to the polarity of the coefficient of the tap position t inthe opposite direction as the coefficient of the tap position t (shiftscoefficients with the same extreme value as the direction of the extremevalue which the coefficient that has been operated has, in the samedirection as the coefficient which has been operated, and shiftscoefficients with a different extreme value as the direction of theextreme value which the coefficient that has been operated has, in theopposite direction as the coefficient which has been operated), so as toaffect change such that the total of coefficient values is 1, maintainedthe equilibrium of the overall distribution balance. In the following,the model wherein coefficient values are changed while maintaining theequilibrium of the overall balance as shown in FIG. 51 will be calledthe equilibrium model. Due to effecting such changes, the equilibriummodel functionally approximates (is equivalent to) a HPF. (High PassFilter) or LPF (Low Pass Filter).

[0616] Now, a case has been described above wherein the coefficientvalue with a positive value has been raised in an equilibrium model, butin the event that the coefficient value with a positive value islowered, for example, i.e., change in the negative direction, positivevalues are changed in the negative direction, and coefficient valueswith negative values are changed in the positive direction. Further, inthe event that coefficient values with negative values are raised,coefficient values with positive values are changed in the negativedirection, and negative coefficients are changed to a positive directionwhile in the event that coefficient values with negative values arelowered, positive coefficient values are changed in a positivedirection, and negative coefficient values are changed in the negativedirection. With equilibrium model, in all cases, the values ofcoefficients are changed in the direction whereby the equilibrium of theoverall balance is maintained.

[0617] As described above, in the event that the amount of change of thechanged coefficient value is greater than the threshold value S11, thechange processing unit 332 changes coefficient values corresponding toother taps by the spring model shown in FIG. 50, and in the event thatthe amount of change of the coefficient value is smaller than thethreshold value S11, changes the coefficient value of coefficientscorresponding to other taps by the equilibrium model shown in FIG. 51.This is since in the event that the amount of change of one coefficientis great, changes to maintain the overall balance are unnatural from theperspective of overall balance of each of the coefficient values, sothis spring model is employed, and in the event that the amount changesare small, the effect that the change in coefficients has on the overallbalance is small, so change is made so as to maintain overall balance.

[0618] Now, models for changing the coefficients other than the changedcoefficient values by operating unit 202 being operated are notrestricted to these, and any is sufficient as long as the total ofcoefficient values is 1 as a whole. Also, in the above-described case,models for changing other coefficient are switched over according to themagnitude of amount of change of a coefficient which is changed by theoperating unit 202 being operated, but the model for changing othercoefficients may be fixed.

[0619] Next, the image optimizing processing of the optimizing device301 shown in FIG. 41 will be described with reference to the flowchartshown in FIG. 52. Now, this image optimizing processing is made up ofcoefficient changing processing and mapping processing. The mappingprocessing is the same as the mapping processing described with FIG. 35and FIG. 38, so only the coefficient changing processing will bedescribed here.

[0620] In step S291, the changing processing unit 332 of the coefficientchanging unit 322 (FIG. 42) judges whether or not operating signals foroperating coefficient values have been input from operating unit 202.That is, in the event that the user deems the image displayed on thedisplay unit 203 to be something matching his/her preferences, mappingprocessing is executed with a coefficient set stored in the coefficientmemory 321 (FIG. 41) here, but in the event that judgment is made thatthis does not match preferences, operations are performed to change thecoefficient set stored in the coefficient memory 321 to be used formapping processing.

[0621] For example, in step S291, in the event that judgment is madethat operating signals for operating coefficients have been input, i.e.,in the event that operating unit 202 has been operated so as to changethe coefficient value of one of the coefficient stored in thecoefficient memory 321, the flow proceeds to step S292.

[0622] In step S292, the change processing unit 332 controls thecoefficient read/write unit 331 to read out coefficient sets stored inthe coefficient memory 321, and the flow proceeds to step S293. In stepS293, the change processing unit 332 judges whether or not thecoefficient value input as operating signals is a change equal to orgreater than a predetermined threshold value S11 in comparison to thevalue contained beforehand in the coefficient set. For example, in theevent that judgment is made in step S293 that the change between thevalue input as operating signals and the value in the coefficient setstored in the coefficient memory 321 is equal to or greater than thethreshold value S11, the processing then proceeds to step S294.

[0623] In step S294, the change processing unit 332 changes the valuesof the coefficient contained in the coefficient set with the springmodel as shown in FIG. 50, and the processing then proceeds to stepS295.

[0624] On the other hand, in the event that judgment is made in stepS293 that the change between the value input as operating signals andthe value in the coefficient set stored in the coefficient memory 321 isnot equal to or greater than the threshold value S11, the processingthen proceeds to step S296.

[0625] In step S296, the change processing unit 332 changes the value ofthe coefficients contained in the coefficient set with the equilibriummodel as shown in FIG. 51, and the processing then proceeds to stepS295.

[0626] In step S295, the change processing unit 332 controls thecoefficient read/write unit 331 so as to overwrite and store the changedcoefficient set values to the coefficient memory 321, the processingreturns to step S291, and subsequently, the same process is repeated.

[0627] In the event that judgment is made in step S291 that theoperating values have not been operated, i.e., in the event that theuser has judged that the image displayed on the display unit 203 matcheshis/her preferences, the flow returns to step S291, and subsequently,the same processing is repeated.

[0628] Due to the above coefficient changing processing, the user canchange coefficient sets used for mapping processing, thereby executingoptimal processing for the user. Now, changing the values of thecoefficients of the coefficient set means that the “contents ofprocessing” of the mapping processing by the mapping processing unit 311are changed.

[0629] Also, with the coefficient changing processing in FIG. 52, in theevent that the magnitude of change of the coefficient is equal to orgreater than a predetermined threshold value S11, all coefficient valuesin the coefficient set are changed with the spring model according tothe value of the coefficient that has been operated, and in the eventthat this is smaller than the threshold value S11, all coefficientvalues in the coefficient set are changed with the equilibrium model, sothat algorithm for changing the coefficient set changes. Accordingly, atthe processing unit 311 of optimizing device 301 shown in FIG. 41 aswell, the “contents of processing”, and further, the “structure ofprocessing” is also changed following user operations, with optimalsignal processing for the user being performed thereby.

[0630] Also, as described above, in the event that the coefficients setstored in the coefficient memory 321 is obtained by least-N′th-powererror, an arrangement may be made wherein, for example, coefficientssets corresponding to multiple exponents N are stored in the coefficientmemory 321 beforehand, with the coefficient changing unit 322 changingthe coefficient set corresponding to the specified exponent N, followingoperating signals from the operating unit 202 based on user operations.In this case, the coefficient sets stored in the coefficient memory 321are changed into those which are generated by the least-N′th-power errormethod corresponding to the exponent N input from the operating unit 202based on user operations i.e., changed to the coefficient set generatedby a different coefficient set generating algorithm, so it can be saidthat the “structure of processing” has been changed.

[0631] Next, an embodiment wherein an internal information generatingunit 371 is provided to optimizing device 301 shown in FIG. 41 will bedescribed with reference to FIG. 53. Note that in FIG. 53, this is thesame as to optimizing device 301 shown in FIG. 41 except for the pointthat the internal information generating unit 371 has been provided.

[0632] The internal information generating unit 371 reads outcoefficient set stored in the coefficient memory 321 for example, asinternal information of the processing unit 311, converts the set toimage signals which can be displayed on the display unit 203, thenoutputs to the display unit 203, and displays.

[0633] Next, the image optimizing processing of optimizing device 301shown in FIG. 53 will be described with reference to the flowchart inFIG. 54. This image optimizing processing is also made up of coefficientchanging processing and mapping processing in the same way as the imageoptimizing processing which the optimizing processing 301 shown in FIG.41 performs, but the mapping processing is the same as the mappingprocessing described with FIG. 35 and FIG. 38, so only the coefficientchanging processing will be described here.

[0634] With the coefficient changing processing, processing the same ofthat in steps S291 through S295 in FIG. 52 is performed in steps S311through S315, respectively.

[0635] In step S315, as with the case in step S295 in FIG. 52, followingthe post-conversion coefficient set being stored in the coefficientmemory 321, flow proceeds to step S317, where the internal informationgenerating unit 371 reads out the coefficient values of the coefficientset stored in the coefficient memory 321, converts these to imagesignals which can be displayed on the display unit 203, outputs to thedisplay unit 203, and displays. At this time, the display unit 203 maydisplay the coefficient values of coefficient sets with a form such as a3-dimensional distribution diagram such as shown in FIG. 39, or a2-dimensional distribution diagram such as shown in FIG. 40, forexample.

[0636] Following the processing step is 317, the flow returns to stepS311, and subsequently, the same processing is repeated.

[0637] According to the coefficient changing processing in FIG. 54, thevalues of the coefficient set stored in the coefficient memory 321 aredisplayed as internal information, so the user can operate the operatingunit 202, while viewing the coefficient set, so as to obtain acoefficient set which executes processing that is optimal for the user.

[0638] Also, with the sum of products computing unit 252 of the mappingprocessing unit 222 (FIG. 31), output signals can be obtained by acomputing a higher order expression of an order of two or higher,instead of the linear expression in Expression (39).

[0639] Next, a configuration example of an optimizing device 401 whichextracts telop portions from image signals as input signals will bedescribed with reference to FIG. 55.

[0640] The features detecting unit 411 of optimizing device 401 detectstwo types of features that have been specified with regard to each ofthe pixels of the image signals serving as input signals, based onoperating signals input from an operating unit 402, and outputs bedetected features information to a processing deciding unit 412. Also,the features detecting unit 411 stores the image signals serving asinput signals in an internal buffer 421 until the telop is extractedfrom the input image signals, and outputs the image signals to theprocessing unit 413. The operating unit 402 is the same as the operatingunit 202 shown in FIG. 41 and FIG. 53. Note that the features detectingunit 411 is not restricted to only detecting two types of features thathave been specified with regard to each of the pixels of the imagesignals serving as input signals, and may, for example, simultaneouslydetect multiple types of features, and output two types of specifiedfeatures from these, or simultaneously detect two or more types offeatures and simultaneously output these.

[0641] The processing deciding unit 412 decides processing which a laterprocessing unit 413 will perform on the image signals, in increments ofpixels, for example, based on the features input from the featuresdetecting unit 411, and outputs the decided processing contents to theprocessing unit 413.

[0642] The processing unit 413 subjects the image signals serving asinput signals read out from the buffer 421 to the processing of theprocessing contents input from the processing deciding unit 412 inincrements of pixels, outputs this to the display unit 403, anddisplays.

[0643] Next, the configuration of the features detecting unit 411 willbe described with reference to FIG. 56. The buffer 421 of the featuresdetecting unit 411 temporarily stores the image signals which are theinput signals, and supplies these to the processing unit 413. Thefeatures extracting and 422 extracts the two types of features selectedby the features selecting unit 423 from the image signals serving as theinput signals, and outputs these to the processing deciding unit 412.The feature selecting unit 423 supplies information for specifying thefeatures to be extracted from the input signals, to the featuresextracting unit 422, based on operating signals input from operatingunit 402. Features which can be selected are, for example, luminancevalues for each of the pixels of the image signals, Laplacian, Sobel,inter-frame of indifference, inter-field difference, the backgrounddifference, and values obtained from the features within a predeterminedrange (sum, average, dynamic range, maximum value, minimum value,central value, or dispersion, etc.), and so forth, but may be otherfeatures as well.

[0644] Next, the configuration of the processing deciding unit 412 willbe described with reference to FIG. 57. The features recognizing unit431 of the processing deciding unit 412 recognizes the multiple types offeatures input from the features detecting unit 421, and outputs, alongwith information indicating the type of features recognized, thefeatures themselves, to the processing contents deciding unit 432. Theprocessing contents deciding unit 432 decides the contents of processingset for each of the features beforehand stored in the processingcontents database 433, based on the information indicating the type offeatures input from the features recognizing unit 431, and the featuresthemselves, and outputs decided processing contents to the processingunit 413.

[0645] Next, the configuration of the processing unit 413 will bedescribed with reference to FIG. 58. The processing content recognizingunit 441 of the processing unit 413 recognizes the processing contentsinput from the processing deciding unit 412, and instructs theprocessing executing unit 442 to execute processing that has beenrecognized. The processing executing unit 442 subjects the input signalsinput via the buffer 421 to the specified processing, based on theinstructions for each pixel from the processing contents recognizingunit 441, converts these into image signals which can be displayed onthe display unit 202 and outputs to the display unit 403, and displays.

[0646] Note that the optimizing device 401 shown in FIG. 55 is forextracting a telop from image signals, so the processing contents areprocessing whether or not to extract each pixel as a telop portion (todisplay or not), but may be made to subject the image signals to otherprocessing as well, and this may be processing wherein input signals areoutput without change with regard to pixels recognized as being a telopportion, while pixels recognized as being other than telop portions arenot output. Also, the image input as input signals will be described asa single frame still image, to facilitate description. However, theoptimizing device 401 in FIG. 55 is also applicable to moving images.

[0647] Next, the telop extracting optimizing processing with theoptimizing device shown in FIG. 55 will be described, with reference tothe flowchart in FIG. 59.

[0648] In step S331, the features extracting unit 422 of the featuresdetecting unit 411 judges whether or not two types of features have beenselected by the features selecting unit 423, and repeats the processinguntil these are selected. That is, the processing in step S331 isrepeated until information indicating the features selected by thefeatures selecting unit 423 is input to the features extracting unit 422based on operating signals corresponding to the type of features inputby the user operating the operating unit 402. For example, in the eventthat judgment is made that information for selecting the features hasbeen input from the features selecting unit 423, i.e., in the event thatjudgment is made that the user has operated operating unit 402 andselected two types of features, the processing proceeds to step S332.

[0649] In step S332, the features extracting unit 422 extracts the twotypes of selected features from the image signals which are the inputsignals, for each pixel, and outputs these to the processing decidingunit 412. At this time, the image signals which are the input signalsare stored in the buffer 421.

[0650] In step S333, the processing deciding unit 412 decides theprocessing contents based on the two types of input-features, for eachof the pixels, and outputs these to the processing unit 413. In moredetail, the features recognizing unit 431 identifies the two types ofinput features, and outputs the identified types of features and thefeatures themselves, to the processing deciding unit 412. Further, theprocessing deciding unit 412 decides the processing contents from thetwo types of features input for each of the pixels. More specifically, atable called an LUT (Look Up Table) correlating, for each of acombination of two arbitrary types which are a feature A and feature B(feature A, feature B), each value of feature A and feature B withprocessing contents regarding pixels having the features A and B (inthis case, information whether or not this is a telop), is stored in theprocessing contents database 433, with the processing contents decidingunit 432 making reference to the LUT based on the combination of the(feature A, feature B) of the pixel of interest which is to be nowprocessed, and decides the corresponding processing, i.e., whether ornot to process as a telop, and outputs this to the processing unit 413.Also, this LUT is generated by extracting multiple features fromtelop-only images beforehand, for example, and correlating withinformation that the combination is a telop. Details of the LUT will bedescribed later.

[0651] In step S334, the processing unit 413 processes the image signalsserving as input signals input via the buffer 421, following theprocessing contents input from the processing deciding unit 412,converts these into image signals which can be displayed on the displayunit 403, outputs to the display unit 403, and displays. In furtherdetail, the processing contents recognize unit 441 of the processingunit 413 recognizes the processing contents input from the processingdeciding unit 412, and instructs the processing executing unit 442 toexecute processing that has been decided with regard to thecorresponding pixels. The processing executing unit 442 reads out theimage signals serving as a input signals that are stored in the buffer421, executes processing corresponding to each of the pixels, convertsthese into image signals which can be displayed on the display unit 403,outputs to the display unit 403, and displays.

[0652] In step S335, the features detecting unit 411 judges whether ornot a telop is deemed to be extracted. That is to say, in the event thatthe user views the image displayed on the display unit 403 and does notjudge that a telop is being extracted, the operating unit 402 isoperated so that the combination of features is changed again and telopextraction processing is attempted. Upon operating signals from theoperating unit 402 corresponding to this operation being input, theprocessing returns to step S331, and the subsequent processing isrepeated.

[0653] On the other hand, in the event that judgment is made accordingto the subjectivity of the user that a telop is being extracted,operating signals indicating completion of the processing are input tothe features detecting unit 421 by the user operating the operating unit402, and the processing ends at this time.

[0654] That is to say, the processing of steps S331 through S335 isrepeated until the user can judge that a telop is being extracted byviewing the image displayed on the display unit 403, due to theabove-described processing, whereby a combinations of features optimalfor the user can be set, and telops can be extracted from image signalsserving as input signals. Now, in the above described, two types offeatures are used to decide the processing contents, but an arrangementmay be made wherein the processing contents are decided by features ofthe number of types other than that number. Also, an arrangement may bemade wherein, in the processing in step S331, combinations of multiplefeatures are sequentially switched over in a predetermined order byoperating signals corresponding to a predetermined operations made bythe user at the operating unit 402 (e.g., button operations instructingup or down, etc.) thereby allowing the user to switch over and input thefeatures without particularly giving consideration to the type offeatures.

[0655] In the above processing, the type of features detected by thefeatures detecting unit 411 is changed at the processing unit 413according to operations of the operating unit 402 by a user, so that thetelop is detected. The changing of type of features detected at thefeatures detecting unit 411 indicates change in the processing contentsdeciding algorithm at the processing deciding unit, so it can be saidthat at the features detecting unit 411 as well, the “structure ofprocessing” is changed.

[0656] Also, as described above, detection of various types of featurescan be made at the features detecting unit 411, but some of the featuresrequire setting of parameters such as filter coefficients and the likein order to detect, such as Laplacian, for example. Parameters fordetecting features can be made changeable corresponding to operations ofoperating unit 402, but according to this parameter changing, the typesof features themselves being detected at the features detecting unit 411do not change, but the values of the features being detected do change.Accordingly, it can be said that the change of parameters for detectingthese features is a change of the “contents of processing” of thefeatures detecting unit 411.

[0657] Next, the configuration of an optimizing device 501 wherein theoptimizing device 401 shown in FIG. 55 is provided with an internalinformation generating unit 511, with reference to FIG. 60. Theoptimizing device 501 shown in FIG. 60 is basically the same inconfiguration as the optimizing device 401 shown in FIG. 55, other thanthe internal information generating unit 511 being provided.

[0658] The internal information generating unit 511 of the optimizingdevice 501 extracts, as internal information, features collectinginformation output from the features selecting unit 423 of the featuresdetecting unit 411, for example, and displays the type of featurescurrently selected on the display unit 403.

[0659] Now, the telop extracting optimizing processing performed by theoptimizing device 501 will be described with reference to flowchart inFIG. 61.

[0660] Note that this processing is basically the same processing as thetelop extraction optimizing processing by the optimizing device 401shown in FIG. 56, described with reference to the flowchart in FIG. 59,and differs in the point that processing for displaying the informationindicating the type of features that has been selected, has been added.

[0661] That is to say, in step S341, the features extracting unit 422 ofthe features detecting unit 411 judges whether or not two types offeatures had been selected by the features selecting unit 423, andrepeats the processing until they are selected. For example, in theevent that judgment is made that information for selecting features hasbeen input from the features selecting unit 423, i.e., in the event thatjudgment is made that the user has operated the operating unit 402 andselected two types of features, the processing proceeds to step S342.

[0662] In step S342, the internal information generating unit 511extracts information indicating the two types of features that have beenselected from the features selecting unit 423, and displays the name ofthe types of the two features that have been selected, on the displayunit 403.

[0663] Subsequently, processing that is the same as that in steps S332through S335 is performed in steps S343 through S346, respectively.

[0664] According to the processing in FIG. 61, the types of featurescurrently selected, which is internal information relating to theprocessing of the features detecting unit 411, is displayed (presented),so the user can set a combination of features optimal for preciselyextracting telop from the image signals serving as input signals, whileunderstanding the types of features currently being selected.

[0665] Now, the internal information generating unit 511 is capable of,for example, generating a distribution of values of two types offeatures detected by the features detecting unit 411 with regard to eachof the pixels, as internal information, and displaying this on thedisplay unit 403, as indicated in the later-described FIG. 65 and FIG.67.

[0666] Also, as described above, in the event of changing the parametersfor detecting features according to operations of operating unit 402,the internal information generating unit 511 can display (present) theparameters on the display unit 403, as internal information.

[0667] Next, a configuration example of an optimizing device 601arranged so as to be provided with an internal information generatingunit 611 which generates internal information from the processingdeciding unit 412, instead of the internal information generating unit511 shown in FIG. 60, will be described with reference to FIG. 62.

[0668] The optimizing device 601 shown in FIG. 62 is configured in thesame way as the optimizing device 501 in FIG. 60 other than the internalinformation generating unit 611 has been provided instead of theinternal information generating unit 511 shown in FIG. 60.

[0669] The internal information generating unit 611 generates, asinternal information, a distribution diagram (e.g., FIG. 65, FIG. 67) ofpixels subjected to a telop extraction at the time of two types offeatures serving as axes, and pixels not subjected to telop extraction,based on the processing contents decided by the processing contentsdeciding unit 432 of the processing deciding unit 412, and the two typesof features actually extracted, which is displayed on the display unit403.

[0670] Next, the telop extraction optimizing processing by theoptimizing device 601 shown in FIG. 62 will be described with referenceto the flowchart in FIG. 63.

[0671] Note that this processing is basically the same processing as atelop extraction optimizing processing performed by the optimizingdevice 501 shown in FIG. 60, which has been described with reference tothe flowchart in FIG. 61, except for the point that processing wherein adistribution regarding whether or not pixels have been subjected totelop extraction with the two types of selected features as axes, isdisplayed.

[0672] That is to say, in step S351, the features extracting unit 422 ofthe features detecting unit 411 judges whether or not two types offeatures have been selected by the features selecting unit 423, and theprocessing is repeated until these are selected. For example, in theevent that judgment is made that information has been input from thefeatures selecting unit 423 for selecting features, i.e., in the eventthat judgment is made that the user has operated the operating unit 402and selected two types of features, the processing proceeds to stepS352.

[0673] In step S352, the features extracting unit 422 extracts two typesof selected features from the image signals serving as input signals foreach of the pixels, and outputs these to the features deciding unit 412.At this time, the image signals which are the input signals are storedin the buffer 421.

[0674] In step S353, the processing deciding unit 412 decides processingcontents for each of the pixels based on input two types of features,and outputs this to the processing unit 413.

[0675] In step S354, the processing unit 413 processes the image signalswhich are the input signals read out from the buffer 421 following theprocessing contents input from the processing deciding unit 412,converts these to image signals that can be displayed on the displayunit 403, outputs to the display unit 403, and displays.

[0676] In step S355, the internal information generating units 611generates, as internal information, a distribution diagram plotting theprocessing contents decided by the processing contents deciding unit ofthe processing deciding unit 412, with the two types of features asaxes, and displays this on the display unit 403.

[0677] In step S356, the features detecting unit 411 judges whether ornot a telop is being extracted. In step S356, in the event thatoperating signals corresponding to user operations are being input fromthe operating unit 202, and it is deemed that there is no telopextraction, the processing returns to step S351, and the subsequentprocessing is repeated.

[0678] On the other hand, in the event that in step S356, operatingsignals indicating ending of the processing are input to the featuresdetecting unit 421 by the user operating the operating unit 402, theprocessing ends.

[0679] That is, let us say that image signals which are input signalssuch as shown in FIG. 64, are input, for example. In FIG. 64, “TITLEABC” is displayed as a telop in the center of the diagram, as to thebackground image (here, the portion which is not a telop).

[0680] In step S355, the internal information generating unit 611displays the distribution indicating whether or not there has beenextraction as a telop, with the two types of features detected from theimage signals such as shown in FIG. 64 as respective axes, as a2-dimensional distribution diagram such as shown in FIG. 65, forexample.

[0681] In the example in FIG. 65, Laplacian and into-frame of differenceare the two types of features that have been selected, with the circlesin the figure indicating pixels that have been extracted as a telop, andthe X's indicating pixels regarding which there has been no telopextraction. With the example in FIG. 65, pixels with telop extractionand pixels with no telop extraction are borderless in the distribution(the distribution between telop pixels and non-telop pixels is notdivided). In the case of such a distribution, the telop is often notbeing extracted from the background image, and as shown in FIG. 66 forexample, is in a state with borders 621 and 622 being formed around thetelop portions, instead of the telop portions themselves being borders.

[0682] In such cases, the user will judge that the telop is not beingextracted, and the processing in steps S351 through S356 is repeated. Inthe event that the selected features due to repetition of thisprocessing are Laplacian sum (17 pixels×17 pixels) (the sum of eachLaplacian of the pixels within the 17 pixel×17 pixel range centered onthe pixel of interest) and luminance DR (the dynamic range of pixelswithin the 17 pixel×17 pixel range centered on the pixel of interest,for the luminance value of each pixel), distribution diagram isgenerated such as shown in FIG. 67. In this case, in FIG. 67, thedistribution of pixels extracted as telop and the distribution of pixelsnot extracted as telop are visually separated. This indicates that thetelop in the input image and other parts are divided into distributionin the event that the combination of the Laplacian sum and luminance DRis selected as features. Upon such a distribution being obtained, theLaplacian sum and luminance DR which are the features detected from eachof the pixels are subjected to a threshold processing and the like,thereby allowing the telop portion to be precisely extracted, as shownin FIG. 68.

[0683] Thus, due to the above-described processing, the processing insteps S351 through S356 is repeated until the user can judge by viewingthe image displayed on the display unit 403 and the 2-dimensionaldistribution with the two selected types of features as axes that thetelop is being extracted, so that the distribution of pixels detected tobe telop from the features serving as internal information relating tothe processing of the processing deciding unit 412, and backgroundpixels, is displayed, whereby the user can operate the operating units402 so as to precisely extract the telop from the image signal which arethe input signals, while understanding the distribution of pixelsdetected to be a telop from the features, and background pixels, therebysetting a combination of features optimal for the user.

[0684] Next, a description will be made regarding the configuration ofan optimizing device 701 provided with a processing deciding unit 711instead of the processing deciding unit 412 of optimizing device 601shown in FIG. 62, with reference to FIG. 69. With optimizing device 701,configurations are the same as those of optimizing device 601 shown inFIG. 62, except that the processing deciding unit 711 is provided instep of the processing deciding unit 412.

[0685] The processing deciding unit 711 changes the contents of the LUTin the processing contents database 433 based on operating signals fromthe operating unit 702 (the same as the operating unit 402), and alsodecides processing for the later processing unit 413 to perform on imagesignals based on features input from the features detecting unit 411,and outputs the processing contents to the processing unit 413.

[0686] The processing deciding unit 711 will next be described withreference to FIG. 70. The basic configuration is the same as theprocessing deciding unit 412 shown in FIG. 50, but processing contentsdeciding unit 721 is provided instead of the processing contentsdeciding unit 432. The processing contents deciding unit 721 changes theLUT wherein is decided the processing contents for each of thecombinations of two types of features stored in the processing contentsdatabase 433, based on operating signals input from the operating unit702. In more detail, the LUT is set such that the features of pixels inan area specified as a telop by the operating unit 202 what is deemed tobe a telop in the state of the image signals which are the input signalsbeing displayed without change, and other areas are processed as beingother than the telop.

[0687] Following the LUT been changed, the processing contents decidingunit 721 decides the processing contents set beforehand for each of thefeatures that are stored in the processing contents database 433, basedon the information for identifying the features input from the featuresrecognizing unit 431, and the features themselves, and outputs thedecided processing contents to the processing unit 413.

[0688] Next, the telop extraction optimizing processing by theoptimizing device 701 shown in FIG. 69 will be described with referenceto the flowchart in FIG. 71.

[0689] In step S361, image signals which are the input signals aredisplayed on the display unit 403 without change. More specifically, thebuffer 421 of the features detecting unit 411 receive the image signalsserving as input signals and store these, and the processing unit 413reads out the image signals which are the input signal stored in thebuffer 421 without change, and outputs these to the display unit 403without any processing, where they are displayed.

[0690] In step S362, the processing contents deciding unit 721 of theprocessing deciding unit 711 judges whether or not the telop andbackground have been instructed from the operating unit 702. That is,let us say that, for example, unprocessed image signals such as shown inFIG. 72 are displayed in the processing in step S361. At this time, theuser operates a pointer 741 via the operating unit 702 (by dragging,clicking, etc.), to specify a general telop portion by a range 742 orthe like, and as shown in FIG. 73, repeats the processing until thetelop portions 752 and the background portion 751 outside the range 742in FIG. 72 are specified, and that the point that the telop and thebackground portion of specified, the specified perspective correspondingpixel positions are stored in built-in memory (not shown), and theprocessing proceeds to step S363.

[0691] In step S363, the features selecting unit 423 of the featuresdetecting unit 421 judges whether or not operating signals for selectingthe predetermined two types of features have been input from theoperating unit 202, the processing is repeated until the predeterminedtwo types of features are specified, and upon the predetermined twotypes of features being selected, the processing proceeds to step S364.

[0692] In step S364, the features extracting unit 422 extracts the twotypes of features selected from the input signals, based on informationfor selecting the types of features a input to the features selectingunit 423, and outputs these to the processing deciding unit 711.

[0693] In step S365, the internal information generating units 611generates a 2-dimensional distribution diagram with the two types offeatures as axes thereof, based on the two types of features input tothe processing deciding unit 711 and the information of pixel positionspecified for the telop and the background, and in step S366, displaysthe 2-dimensional distribution diagram on the display unit 403. Infurther detail, the features recognizing unit 431 of the processingdeciding unit 711 recognizes the type of features, outputs informationindicating the type, and the features themselves, to the processingcontents deciding unit 721, and the processing contents deciding unit721 outputs information indicating the pixel positions specified for thetelop and background to the internal information generating 611, inaddition to the information indicating features and the types thereof,so that the internal information generating unit 611 generates2-dimensional distribution diagram of the features such as shown in FIG.74, for example, based on the information indicating the pixel positionsspecified for the telop and background. That is to say, with the examplein FIG. 74, an example is illustrated wherein Laplacian and inter-frameddifference are selected as features, with the circles in the figureindicating telop pixels, and the X's in the figure indicating pixelspecified as being the background. With the 2-dimensional distributiondiagram in FIG. 74, for example, with regard to a particular pixel, theLaplacian is detected at a value of X, the inter-frame difference isdetected at a value of Y and in the event that that pixel has beenspecified as telop, a circle is displayed at the position of (X, Y) onthe 2-dimensional distribution, while in the event that background hasbeen specified, an X is displayed at that same position.

[0694] In step S367, the processing contents deciding unit 721 judgeswhether or not operating signals indicating that judgment has been madethat the telop and background have been separated have been input. Thatis, for example, in the case of a 2-dimensional distribution such asshown in FIG. 74, the distribution of circles indicating the telop andthe distribution of X's indicating the background cannot be said to becompletely separated, so as a matter of course, the telop cannot beexpected to be extracted on the display screen, and in many cases thebackground and telop are in the state of not being separated such as, asshown in FIG. 73 for example, a border 753 is generated separate to thebackground portion 751 and the telop portions 752. In such cases whereinthe user judges that the telop is not extracted, at the point of theuser attempting to change the features again, the operating unit 702outputs operating signals indicating that separation has not beenachieved to the processing contents deciding unit 721 of the processingdeciding unit 711, in accordance with operations made by the user. Inthis case, in step S367, judgment is made that there have not beenoperating signals indicating judgment that the telop and background havebeen separated, so the processing returns to step S363, and thesubsequent processing is repeated. This processing means that two typesof features are selected again.

[0695] Also, in the event that as shown in FIG. 75, for example, thecircles indicating the telop portion and the X's indicating thebackground are separated to a certain extent by the processing in stepS366, the operating unit 702 is operated by the user to output operatingsignals indicating that separation has been achieved, to the processingcontents deciding unit 721 of the processing deciding unit 711. In thiscase, judgment is made that operating signals indicating a judgment thatthe telop and background have been divided has been input in step S367,and the processing proceeds to step S368.

[0696] In step S368, the processing contents deciding unit 721 judgeswhether or not the telop portion has been instructed on the2-dimensional distribution. That is, as shown in FIG. 75, judgment ismade whether or not operating signals specifying the range 761, forexample, by the operating unit 702, has been input as the range where agreat number of circles indicating the telop are distributed on thedisplayed distribution, with the pointer 741, the processing is repeateduntil the range specified, and upon judgment being made that this hasbeen specified, the processing proceeds to step S369.

[0697] In step S369, the processing contents deciding unit 721 changesthe processing contents: the LUT indicating whether or not there hasbeen telop extraction corresponding to combinations of the features,based on operating signal specified in the range 761 shown in FIG. 75that has been input by the operating unit 702, and decides theprocessing context according to the changed LUT which are output to theprocessing unit for 13, while the processing unit 413 follows the inputprocessing contents to extract the telop from the image signals servingas the input signal that had been input via the buffer 421, and todisplay these owned the display unit 403. In further detail, theprocessing contents deciding unit 721 updates the LUT in the processingcontents database 433 so as to extract the combination of two types offeatures corresponding to the pixels distributed in a specified range astelop, based on information indicating a range on the 2-dimensionaldistribution such as shown in FIG. 75, that has been input from theoperating unit 702, and further decides the processing contents for eachpixel following the updated LUT, which are output to the processing unit413.

[0698] In step S370, the processing contents deciding unit 721 judgeswhether or not judgment has been made that the telop has been extracted.That is, judgment is made whether the user judges the telop to beextracted or judges the telop to not be extracted. For example, as shownin FIG. 76, in the state that the output images displayed on the displayunit 403 by the processing in step S369, the boundaries 771 and 772between the telop portion and background portion are not the telopitself, so it cannot be said that the telop is completely extracted.Accordingly, in the event that the telop is judged to be not extractedfrom the perspective of the user, the user operates the operating unit702 to input operating signals indicate that the telop has not beenextracted. Upon receiving these operating signals, the processingcontents deciding unit 721 of the processing deciding unit 711 makesjudgment in step S730 that the telop is not extracted, and theprocessing proceeds to step S731.

[0699] In step S371, judgment is made regarding whether or not tore-specify the range for the telop on the 2-dimensional distribution,and in the event that judgment is made to that not re-specifying therange for the telop on the 2-dimensional distribution has been selected,the processing returns to step S368, and the subsequent processing isrepeated. On the other hand, in step S370, in the event thatre-specifying the range for the telop has been selected by theprocessing in step S371, for example, a range 781 wherein the portionwhere the distribution of circles indicating the telop exists has beennarrowed down as compared to the range 761 in FIG. 75 (a range narrowingdown the portion containing a great number of circles extracted more astelop), as shown in FIG. 77, is set in step S368. That is, in FIG. 75,even in the event of setting the range 761 on the features distributionas a telop portion, consequently, the boundaries 771 and 772 between thetelop portion and background portion are not the telop itself, as shownin FIG. 76, so the telop is not completely extracted.

[0700] Accordingly, the user operates the operating unit 702 to set arange 781 which is narrower than the range 761, as a range over thefeatures distribution (as a range of a portion to specify as a telopportion). Setting the range thus narrows down the range on the featuresdistribution to be extracted as a telop, that is to say, the backgroundportion is more readily eliminated. However, ignoring the range to beextracted as telop too much causes the telop itself to not be readilyextracted, so the user searches for an optimal telop extractions stateby repeating such operations while viewing the extracted telop.

[0701] Upon the state being that such as shown in FIG. 78, for example,wherein the telop is extracted, and in the event that the user judgesthat the telop is extracted in step S370, the user operates theoperating unit 702, and operating signals indicating that jet and hasbeen made that the telop has been extracted are input to the processingcontents deciding unit 721 of the processing deciding unit 711, sojudgment is made that the telop is extracted, and the processing ends.

[0702] Also, in the event that judgment is made in step S371 that thisdoes not specified the range of the telop, the processing returns tostep S363, and subsequent processing is repeated.

[0703] Due to such processing, first, the user selects two types offeatures of the telop and background on the image signals which are theinput signals, and, viewing how the selected telop and background form a2-dimensional distribution with the selected features as the two axesthereof, changes (narrows down) the range of the telop on the2-dimensional distribution, or changes the two selected features, andfurther views that extracted telop, and repeats this processing untilthe state is that preferred by the user, thereby a realizing telopextraction processing matching the preferences of the user. In the eventthat judgment is made in the processing in step S362 that the telop andbackground have been instructed, it is sufficient that instructions ofthe range of telop and background serves to subsequently select twotypes of features or generate information serving as a template fornarrowing down the telop on the 2-dimensional distribution, so the rangecan be very general.

[0704] In the above processing, division is made into telop orbackground processing, according to combinations of two types offeatures selected by the user, so this means that the “contents ofprocessing” have been changed by user operations.

[0705] Also, with the above processing, the type of features aredetermined by a two types of features being specified by the operatingunit 402, but an arrangement may be made such as shown in FIG. 79, forexample, wherein the combination of the two types of features can bechanged by a sending instructions of up or down as operating signals bya predetermined operating button on the operating unit 702. That is, anarrangement may be made wherein, as an initial state, processing isexecuted by a combination of features a and b indicated in state A, thennext, instructing down executes processing by the combination offeatures b and c indicated in the state B, and further instructing downexecutes processing with a combination of the features c and d indicatedin the state C, wherein, in the event of specifying up in the state C,this returns to the state B, and in the event of specifying up in thestate B, this returns to the state A due to such an arrangement, theuser can change the features one after another without giving noparticular consideration to the type of features, so the combination offeatures for extracting telop can be effectively narrowed down.

[0706] Next, description will be made regarding a configuration of anoptimizing device 801 provided with the features detecting unit 811capable of generating new features from already-existing features,instead of the features detecting unit 411 of the optimizing device 401in FIG. 55, with reference to FIG. 80. In FIG. 80, the configuration isthe same except for providing the features detecting unit 811 instead ofthe features detecting unit 411 of the optimizing device 401 in FIG. 55.

[0707] Further, the operating unit 802 is the same as the operating unit402.

[0708] The configuration of the features detecting unit 811 shown inFIG. 80 will be described with reference to FIG. 81. With the featuresdetecting unit 811, the buffer 421, and the features extracting unit 422are the same as the features detecting unit 411 shown in FIG. 56. Thisfeature selecting unit 821 controls the features extracting unit 422based on operating information specifying the features input from theoperating unit 802, to extract two types of features that have beenspecified out of the features provided beforehand, and output these tothe processing deciding unit 413, or output features stored in thefeatures database 823 beforehand, to the processing deciding unit 413.In more detail, the features database 823 stores features informationregarding the types of features and the detection methods for thefeatures. The features extracting unit 422 reads out featuresinformation corresponding to the types of features selected by thefeatures selecting unit 821 from the features database 823, and detectsthe selected features from the input signals, following the featuresdetecting method recorded in the features information.

[0709] Features information provided beforehand are luminance values foreach of the pixels of the image signals, Laplacian, Sobel, inter-framedifference (e.g., inter-frame difference of a pixel of interest (x, y)is represented by f_(a)(x, y)=f₀(x, y)−f₁(x, y), wherein f₀(x, y)indicates the current pixel of interest, while f₁(x, y) represents apixel from one frame back on the same spatial position), inter-fielddifference, background difference, derivative values (e.g., thederivative value f_(b)(x, y) of a pixel of interest (x, y) isrepresented by f_(b)(x, y)=4×f₀(x, y)−f₀(x−1, y)−f₀(x+1, y)−f₀(x,y+1)−f₀(x, y−1), wherein f₀(x, y) represents the current pixel ofinterest, and f₀(x−1, y), f₀(x+, y), f₀(x, y+1), and f₀(x, y−1)represent pixels removed by one pixel each in the x and y directions onthe space), and values obtained from the features within a predeterminedrange (sum, average, dynamic range, maximum value, minimum value,central value, or dispersion, etc.), and so forth, but may be others aswell.

[0710] The features altering unit 822 generates new features from thefeatures stored in the features database 823, based on operating signalsinput from the user. In further detail, the features altering unit 822will generates new features information from features information storedin the features database 823, based on operating signals input from theuser, and the features extracting unit 422 extracts new features basedon of this features information.

[0711] Also, saying that there is features information for features A,B, and C, as types of features, stored in the features database 823, anarrangement may be made wherein the DR (dynamic range: of valuegenerated by reading out the value for the feature A for multiple pixelsthat exist at a predetermined position as to the pixel of interest, andobtaining of value which is the difference between the minimum value andmaximum value, for each pixel) of the feature A may be made to befeature information corresponding to a new feature A′, or may be made tobe new features information by similarly obtaining sum, average, dynamicrange, maximum value, minimum value, central value, dispersion, thenumber of pixels having a value equal to or greater than a thresholdvalue (the threshold value may also be set), or a linear combinationbetween multiple features. Note that a linear combination betweenmultiple features is, for example, in the event that the Laplacian for apredetermined number of pixels is X_(a), Sobel is X_(b), and inter-framedifference is X_(c) with a linear combination of these three types offeatures being taken, the coefficients of these three types of featuresmultiplied and summed, i.e., A×x_(a)+B×x_(b)+C×x_(c) is the new featureof this pixel. Here, A, B, and C are coefficients, and in the case oftelop extraction for example, these coefficients can be obtained by thelearning processing when the pixels in the range that has been generallyspecified to for the telop portion by the operating unit 802 as thetutor data, and the multiple features as student data, as described withFIG. 43 and FIG. 44. In FIG. 81, with regard to the features stored inthe features database 823, features A through C indicate the types offeatures extracted by the features extracting unit 422, and features A′through C′ the types of features altered from the features A through Cby the features altering unit 822 (what is actually stored is and thefeatures information of each (the type of features, and informationindicating the detection method thereof)).

[0712] The features information stored in the features database 823 maybe those specified that the timing of being extracted by the featuresextracting unit 422 being stored, maybe those wherein featuresinformation stored in the features database 823 beforehand is alteredand by the features altering unit 822, or may be stored with othermethods.

[0713] Next, the telop extraction optimizing processing by theoptimizing device 801 in FIG. 80 will be described with reference to theflowchart in FIG. 82. In step S381, the features extracting unit 422 ofthe features detecting unit 411 judges whether or not two types offeatures have been selected by the features selecting unit 821, and inthe event that these are not selected, the processing proceeds to stepS386. Also, in the event that, for example, judgment is made thatinformation for selecting features has been input from the featuresselecting unit 423, i.e., in the event that judgment is made that theuser has operated operating unit 802 and selected two types of features,the processing proceeds to step S382.

[0714] In step S382, the features extracting unit 22 extracts the twotypes of selected features from the image signals input signals for eachpixel, and outputs to the processing deciding unit 412. At this time,signals serving as the input signals are stored in the buffer 421.

[0715] In step S383, the processing deciding unit 412 decides theprocessing contents for each pixel based on the input two types offeatures, and outputs to the processing unit 413.

[0716] In step S384, the processing unit 413 processes the image signalswhich are input signals input via the buffer 421 following theprocessing contents input from the processing deciding unit 412,converts these into image signals that can be displayed on the displayunit 403, and outputs to the display unit 403 and displays.

[0717] In step S385, the features detecting unit 411 judges whether ornot the telop is deemed to be extracted. That is, the user views theimage displayed on the display unit 403, and in the event of not judgingthat the telop has been extracted, operates operating unit 802 so as tochange the combinations of features again and attempt telop extractionprocessing. In the event that operating signals corresponding to thisoperation are input from operating unit 802, the processing proceeds tostep S386.

[0718] In step S386, the features altering unit 822 of the featuresdetecting unit 811 judge whether or not altering of the features hasbeen specified, and in the event there is no instruction for alterationof the features, the processing returns to step S381. On the other hand,in the event that the features altering unit 822 judges in step S386that there has been operating signals input from the operating unit 802which instruct alteration of the features, the processing proceeds tostep S387.

[0719] In step S387, the features altering unit 822 judges whether ornot basic features have been specified, and the processing is repeateduntil information specifying the basic features is input. For example,in the event that the operating unit 802 is operated and operatingsignal instructing the feature A is input, judgment is made that afeature based on the feature A in the image signals which is the inputsignals has been input, and the processing proceeds to step S388.

[0720] In step S388, the features altering unit 822 judges whether ornot alteration processing contents have been instructed, and repeats theprocessing until alteration processing contents are instructed. Forexample, in the event that the operating unit 802 is operated andoperating signals instructing DR are input, judgment is made that thealteration processing contents have been specified, and the processingproceeds to step S389.

[0721] In step S389, the features altering unit 822 alters the specifiedfeatures with the specified alteration processing contents to obtain newfeatures, which is stored in the features database 823, and theprocessing returns to step S381. That is to say, in this case, thefeatures altering unit 822 reads the features A out of the featuresdatabase 823 and obtains DR, which is the specified alterationprocessing contents thereof, thereby generating the new features A′,which is stored in the features database 823, and the processing returnsto step S381.

[0722] On the other hand, in the event that judgment is made in stepS385 according to the subjectivity of the user that a telop is beingextracted, operating signals indicating completion of the processing areinput to the features detecting unit 421 by the user operating theoperating unit 802, and the processing ends at this time.

[0723] That is to say, the processing of steps S381 through S389 isrepeated until the user can judge that a telop is being extracted byviewing the image displayed on the display unit 403, due to theabove-described processing, whereby a combinations of features optimalfor the user can be set, and telops can be extracted from image signalswhich are input signals, and further, even more combinations of featurescan be set by increasing the types of features for the user to select,thereby executing processing that is optimal for the user.

[0724] In the above processing, processing contents are decidedaccording to the two types of features which the user has specified, andtelop is extracted from the image signals which are input signals, so itcan be said that the “contents of processing” are changed such thatoutput signals desired by the user are obtained, following operations ofthe user. Also, the two axes of features (the two types of featuresbeing selected) are switched over, while new features are also set (thetypes of features increase), and that algorithm for deciding theprocessing contents (e.g., whether or not to process as a telop) ischanged by the combination of the features, so it can be said that, of“contents of processing”, the “structure of processing” is also changed.

[0725] Also, telop extraction is obtained by switching over the types offeatures to be extracted by trial and error, but with methods usingfeatures in a fixed manner, there has been the need to rewrite theprogram or remake the system itself in order to change the featuresanticipated, so the system has to be remake many times to obtain thealgorithms optimal for telop extraction by discovery, and has been quitedifficult in reality. Conversely, with the optimizing device accordingto the present invention, new features can be extracted in real-time,and the features distribution thereof can be shown, so the trial anderror performed by the user becomes easier, and the possibility offinding features optimal for telop extraction is improved.

[0726] Next, the configuration of an optimizing device 901 arranged byproviding the optimizing device 801 shown in FIG. 80 with the featuresdetecting unit 911 and internal information generating unit 912provided, will be described with reference to FIG. 83. The optimizingdevice 901 shown in FIG. 83 is basically the same as the optimizingdevice 801 shown at FIG. 80 and configuration, except that the featuresdetecting unit 911 is provided instead of the features detecting unit811, and further, an internal information generating unit 912 has beennewly provided.

[0727] Next, the configuration of the features detecting unit 911 willbe described with reference to FIG. 84. The features detecting unit 911is basically the same configuration as the features detecting unit 811in FIG. 81, but is provided with the features selecting unit 921 insteadof the features selecting unit 821, and a features altering unit 922instead of the features altering unit 822. The basic functions are thesame for each, but the features selecting unit 911 differs in thatinformation of that type of selected features is supplied to the ofinternal information generating unit 912. Also, features altering unit922 differs in that image information relating to instructions ofalteration contents are output to the display unit 403.

[0728] The internal information generating unit 912 is the same as theinternal information generating unit 511 shown in FIG. 60. Also, theoperating unit 902 is the same as the operating unit 402.

[0729] Next, the telop extraction optimizing processing with theoptimizing device 901 shown in FIG. 83 will be described, with referenceto the flowchart in FIG. 85. Now, this processing is basically the sameas the processing described with reference to the flowchart in FIG. 82,but differs in that the features selected by the internal informationgenerating unit 912 are displayed.

[0730] That is to say, in step S391, the features extracting unit 422 ofthe features detecting unit 911 judges whether or not two types offeatures have been selected by the features selecting unit 921, and inthe event that these have not been selected, the processing proceeds tostep S397. Also, in the event that judgment has been made to thatinformation for selecting the features have been input from the featuresselecting unit 921, for example, the processing proceeds to step S392.

[0731] In step S392, the internal information generating unit 912extracts the information indicating that types of the two selected typesof features from the features selecting unit 921, and displays the namesof the types of the two selected features on the display unit 403.

[0732] In step S393, the features extracting unit 422 extracts the twotypes of selected features from that image signals which are the inputsignals, for each pixel, and outputs these to the processing decidingunit 412. At this time, the image signals which are the input signalsare stored in the buffer 421.

[0733] In step S394, the processing deciding unit 412 decides theprocessing contents for each pixel based on the two types of featuresthat had been input, and outputs to the processing unit 413.

[0734] In step S395, the processing unit 413 processes the image signalswhich are input signals input via the buffer 421 following theprocessing contents input from the processing deciding unit 412,converts these into image signals that can be displayed on the displayunit 403, outputs to the display unit 403, and displays.

[0735] In step S396, the features detecting unit 411 judges whether ornot the telop is deemed to have been extracted. That is, the user viewsthe image displayed on the display unit 403, and in the event of notjudging that the telop has been extracted, operates the operating unit902 so as to change the combinations of features again and attempt telopextraction processing. In the event that operating signals correspondingto this operation are input from operating unit 902, the processingproceeds to step S397.

[0736] In step S397, the features altering unit 922 of the featuresdetecting unit 911 judges whether or not altering of the features hasbeen specified, and in the event there is no instruction for alterationof the features, the processing returns to step S391. On the other hand,in step S397, in the event that the features altering unit 922 judgesthat there have been operating signals input from the operating unit 902which instruct alteration of the features, the processing proceeds tostep S398.

[0737] In step S398, the features altering unit 922 displays aninstruction screen for alteration contents, such as shown in FIG. 86,for example. In FIG. 86, the basic features display unit 931 is providedto the left side in the figure, where features serving as a basis, whichare features currently stored in the features database 823, aredisplayed. In this case, features A through C and A′ through C′ aredisplayed. Also displayed to the right side thereof is an alterationprocessing content selection space 932. In this case, DR, maximum value,minimum value, central value, sum, dispersion, the number of pixelshaving a value equal to or greater than a threshold value, and linearcombination are displayed, with the space 932 a for setting thethreshold value provided with regard to the number of pixels having avalue equal to or greater than a threshold value, and further, the space932 b for selecting features in the event of selecting the linearcombination. Further, a scale settings space 933 is a displayed forsettings scales of the values. A scale value indicates the rangecentered on the pixel of interest, and is of value indicating the rangeof pixels such as, for example, in the case of detecting DR, setting thenecessary pixels to 3 pixels×3 pixels or 5 pixels×5 pixels, and soforth.

[0738] In step S399, the features altering unit 922 judges whether ornot features serving as a basis have been specified, and repeats theprocessing until information for specifying the features as a basis isinput. For example, in the event that the operating unit 902 is operatedand operating signals instructing the feature A are input, judgment ismade that the feature A has been input as a feature serving as a basisfor the image signals which are input signals, and the processingproceeds to step S400.

[0739] In step S400, the features altering unit 922 judges whether ornot alteration processing contents have been instructed, and theprocessing is repeated until the alteration processing contents areinstructed. For example, in the event that the operating unit 902 isoperated and operating signals instructing DR are input, judgment ismade that the alteration processing contents have been instructed, andthe processing proceeds to step S401.

[0740] In step S401, the features processing unit 922 alters thespecified features with the specified alteration processing contents,stores this in the features database 823, and the processing returns tostep S391.

[0741] On the other hand, in the event that judgment is made accordingto the subjectivity of the user that a telop is being extracted,operating signals indicating completion of the processing are input tothe features detecting unit 421 by the user operating the operating unit902, and the processing ends at this time.

[0742] That is, due to the above-described processing, the user performsoperation while viewing the features serving as a basis and thealteration processing contents, and accordingly can recognize thefeatures whereby optimal processing can be performed. Further,subsequently, features capable of realizing optimal processing for theuser can be immediately specified. Also, the type of features which theuser specifies can be increased by inputting input information necessaryfor generating new features, following the display screen, socombinations of many features can be efficiently set.

[0743] In the above processing, the processing contents are decidedaccording to two types of features that have been specified by the user,the telop is extracted from the image signals which are the inputsignals, so it can be said that the “contents of processing” are beingchanged so that output signals desired by the user can be obtained,following operations made by the user. Also, the two axes of thefeatures (the two types of features that are selected) are switchedover, while new features are also set (the types of features increase),and the algorithm for deciding the contents of processing (e.g., whetheror not to process as a telop) is changed by the combination of thesefeatures, so of the “contents of processing”, the “structure ofprocessing” is also changed.

[0744] Next, the configuration of an optimizing device 1001 which isprovided with an internal information generating unit 1011 instead ofthe internal information generating unit 912 of the optimizing device901 shown in FIG. 83, will be described with reference to FIG. 87. InFIG. 87, the structure is the same as that of the optimizing device 901shown in FIG. 83, other than the internal information generating unit912 being replaced with the internal information generating unit 1011,and further, the processing deciding unit 412 being replaced with theprocessing deciding unit 711 shown in FIG. 69.

[0745] The internal information generating unit 1011 is basically thesame as that internal information generating unit 912, but further readsout information of processing contents decided for each pixel, decidedby the processing contents deciding unit 432 of the processing decidingunit 412, and displays this on the display unit 403 (the functions ofthe internal information generating unit 611 shown in FIG. 69 have beenadded).

[0746] Also, the operating unit 1002 is the same as the operating unit402.

[0747] Next, the telop extraction optimizing processing with theoptimizing device 1001 shown in FIG. 87 will be described with referenceto the flowchart in FIG. 88.

[0748] In step S411, the buffer 421 of the features detecting unit 911receives image signals which are input signals and stores these, and theprocessing unit 413 reads out the image signals which are input signalsstored in the buffer 421 without change, and output these to the displayunit 403 without any processing, where they are displayed.

[0749] In step S412, the processing contents deciding in its 721 of theprocessing deciding unit 711 judges whether are not the telop andbackground have been instructed by the operating unit 1002, theprocessing is repeated until the telop and background are instructed,and upon the telop and background being instructed, the processingproceeds to step S413.

[0750] In step S413, the features extracting unit 422 of the featuresdetecting unit 911 judges whether or not two types of features have beenselected by the features selecting unit 921, and if not selected, theprocessing proceeds to step S421. Also, in the event that judgment ismade that information selecting the features has been input from thefeature selecting unit 921, for example, the processing proceeds to stepS414.

[0751] In step S414, the internal information generating unit 912extracts information indicating the types of the two types of featuresthat have been selected from the features selecting unit 921, anddisplays (presents) names of the types of the two types of features thathave been selected, on the display unit 403.

[0752] In step S415, the features extracting unit 422 extracts theselected two types of features for each of the pixels from the imagesignals which are input signals, and outputs these to the processingdeciding unit 412. At this time, the buffer 421 stores the image signalswhich are input signals.

[0753] In step S416, the processing deciding unit 711 decides theprocessing contents for each of the pixels based on the two types offeatures that have been input, and output these to the processing unit413.

[0754] In step S417, the internal information generating unit 1011generates a 2-dimensional distribution based on the two types offeatures input to the processing deciding unit 711 and information ofthe pixel position specified for the telop and background, with the twotypes of features as the axes thereof, and in step S418, displays the2-dimensional distribution on the display unit 403. In further detail,the features recognizing unit 431 of the processing deciding unit 711recognizes the types of features, and outputs the information indicatingthat the type thereof, and the features themselves to the processingcontents deciding unit 721, the processing contents deciding unit 721outputs this information indicating the pixel positions specified forthe telop in the background, in addition to the information indicatingthe features and the types thereof, to the internal informationgenerating unit 1011, whereby the internal information generating unit1011 generates a 2-dimensional distribution diagram of the features suchas shown in FIG. 74, for example, based on the information indicatingthe pixel positions specified for the telop and background.

[0755] In step S419, the processing unit 413 processes the image signalswhich are input signals, input via the buffer 421, following theprocessing contents input from the processing deciding unit 711,converts these into image signals which can be displayed on the displayunit 403, outputs to the display unit 403, and displays.

[0756] In step S420, the features detecting unit 911 judges whether ornot the telop is deemed to have been extracted. That is, the user viewsthe image displayed on the display unit 403, and in the event of notjudging that the telop has been extracted, operates the operating unit1002 so as to change the combinations of features again and attempttelop extraction processing. In the event that operating signalscorresponding to this operation are input from operating unit 1002, theprocessing proceeds to step S421.

[0757] In step S421, the features altering unit 922 of the featuresdetecting unit 911 judges whether or not altering of the features hasbeen specified, and in the event there is no instruction for alterationof the features, the processing returns to step S413. On the other hand,in step S421, in the event that the features altering unit 922 judgesthat there have been operating signals input from the operating unit1002 which instruct alteration of the features, the processing proceedsto step S422.

[0758] In step S422, the features altering unit 922 displays aninstruction screen for alteration contents(FIG. 86).

[0759] In step S423, the features altering unit 922 judges whether ornot features serving as a basis have been specified, and the processingis repeated until the features serving as a basis have been input. Inthe event that judgment is made that features serving as a basis havebeen input, the processing proceeds to step S424.

[0760] In step S424, the features altering unit 922 judges whether ornot alteration processing contents have been instructed, and theprocessing is repeated until the alteration processing contents areinstructed. In the event that judgment is made that the alterationprocessing contents have been input, the processing proceeds to stepS425.

[0761] In step S425, the features altering unit 922 alters the specifiedfeatures serving as a basis with the specified alteration processingcontents, stores this in the features database 823, and the processingreturns to step S413.

[0762] On the other hand, in the event that judgment is made in stepS420 according to the subjectivity of the user that a telop is beingextracted, operating signals indicating completion of the processing areinput to the features detecting unit 421 by the user operating theoperating unit 1102, and the processing ends at this time.

[0763] That is, due to the above-described processing, the processing insteps S411 through S425 is repeated until the user can judge fromviewing that image displayed on the display unit 403 that the telop isextracted, thereby enabling a combination of features optimal for theuser to be set and the telop to be extracted from the image signalswhich are input signals, and increasing the types of features which canbe selected by the user enables a great number of combinations offeatures to be set, so as to execute processing that is optimal for theuser. Also, an alteration instruction screen necessary for alteringalready-existing features and generating new features is displayed, sothe user can efficiently execute the alteration processing following thedisplay. Also, at this time, the telop extraction processing can berepeatedly executed while changing the features while viewing the stateof separation between telop and background with regard to the featuresof the pixels, so features capable of highly precise telop extractioncan be easily selected.

[0764] Next, the configuration of an optimizing device 1101 providedwith a features detecting unit 1111, processing deciding unit 1112, andoperating unit 1102, instead of the features detecting unit 411,processing deciding unit 412, and operating unit 402, of the optimizingdevice 401 shown in FIG. 55, will be described with reference to FIG.89. In FIG. 89, the configuration is the same as that of the optimizingdevice 401 shown in FIG. 55, other than the features detecting unit1111, processing deciding unit 1112, and operating unit 1102 being newlyprovided.

[0765] Of the configurations of the features detecting unit 411 in FIG.56, features detecting unit 1111 is not provided with the featuresselecting unit 423, with the features extracting unit 422 extracts twotypes of features that have been set beforehand, but otherwise is of thesame configuration.

[0766] The processing deciding unit 1112 stores history informationwhereby the LUT stored in the processing contents database 433 isupdated, and updates the LUT corresponding to the history information.The configuration of the processing deciding unit 1112 will be describedlater with reference to FIG. 90.

[0767] Also, the operating unit 1102 is the same as the operating unit402.

[0768] Now, the configuration of the processing deciding unit 1112 willbe described with reference to FIG. 90. The processing deciding unit1112 in FIG. 90 is of the same configuration of the processing decidingunit 412 other than being provided with the processing contents decidingunit 1121 instead of the processing contents deciding unit 432 of theprocessing deciding unit 412 shown at FIG. 57, and further, havinghistory memory 1122 added thereto.

[0769] The processing contents deciding unit 1121 stores historyinformation of the operations for changing the LUT stored in theprocessing contents database 433, in the history memory 1122, andchanges the LUT based on the history information. Other functions arealso the same as those of the processing contents deciding unit 432 ofthe processing deciding unit 412 shown in FIG. 57, as well.

[0770] Next, the telop extraction optimizing processing by theoptimizing device 1101 shown in FIG. 89 will be described with referenceto the flowchart shown in FIG. 91.

[0771] In step S431, the features extracting unit 411 of the featuresdetecting unit 1111 extracts two predetermined types of features fromthe image signals which are input signals, and outputs these to theprocessing deciding unit 1112. At this time, the buffer 421 stores theimage signals which are input signals.

[0772] In step S432, the processing contents deciding unit 1121 of theprocessing deciding unit 1112 decides the processing contents for eachpixel making reference to the LUT stored in the processing contentsdatabase 433, based on the types of features and the features input fromthe features recognizing unit 431, and outputs these to the processingunit 413.

[0773] In step S433, the processing unit 413 processes the pixelsfollowing the processing contents input from the processing decidingunit 1112, outputs to the display unit 403, and displays.

[0774] In step S434, the processing contents deciding unit 1121 of theprocessing deciding unit 1112 to this whether or not operating signalsfor changing the LUT have been input from the operating unit 1102. Thatis, the user views the image displayed on the display unit 403 andsubjectively judges whether processing which is according to thepreferences of the user is being performed, the user operates theoperating unit 1102 based on the judgement results, so thatcorresponding operating signals are input. For example, in the eventthat processing matching the preferences of the user is not beingperformed (in the event that a screen according to the preferences ofthe user is not being displayed on the display unit 403), a request forchanging the LUT is input for changing the LUT in the processingcontents database 433 of the processing deciding unit 1121.

[0775] In step S434, in the event that operating signals requestingchange of the LUT have been input, i.e., processing according to thepreferences of the user is not being performed, the processing proceedsto step S435.

[0776] In step S435, judgment is made regarding whether or not thechanging processing of the LUT can be executed by auto LUT changingprocessing. Here, manual LUT changing processing and auto LUT changingprocessing exist for changing processing for the LUT. The details ofthat judgment regarding whether or not auto LUT changing processing iscapable will be described later.

[0777] For example, in the event that due to and is made in step S435that auto LUT changing processing is not possible, the processingproceeds to step S436, where manual LUT changing processing is executed.

[0778] Now, along with a description of the LUT changing processing, thedetails of the LUT will be described. The LUT is a table indicatingprocessing contents which are decided for each of the combinations oftwo features, as shown in FIG. 92. In FIG. 92, a case of features A andB as the two types of features is illustrated, and in this example, anexample is shown wherein the features are classified into 8 steps (atotal of 64 classes). Also, in FIG. 92, the features are normalized tovalues of 0 through 1.0, and are divided into the ranges of, with regardto the values V_(a) of the feature A, from the left, 0≦V_(a)<⅛,⅛≦V_(a)<{fraction (2/8)}, {fraction (2/8)}≦V_(a)<⅜, ⅜≦V_(a)<{fraction(4/8)}, {fraction (4/8)}≦V_(a)<⅝, ⅝≦V_(a)<{fraction (6/8)}, {fraction(6/8)}≦V_(a)<⅞, ⅞≦V_(a)≦{fraction (8/8)}, and, with regard to the valuesV_(b) of the feature B, from the top, 0≦V_(b)<⅛, ⅛≦V_(b)<{fraction(2/8)}, {fraction (2/8)}≦V_(b)<⅜, ⅜≦V_(b)<{fraction (4/8)}, {fraction(4/8)}≦V_(b)<⅝, ⅝≦V_(b)<{fraction (6/8)}, {fraction (6/8)}≦V_(b)<⅞,⅞≦V_(b)≦{fraction (8/8)}. In the figure, the processing contents areclassified into the three types of X, Y, and Z, according to thecombinations of features of the ranges, and in the case shown in FIG.92, in the range of 0≦V_(a)≦⅜ and 0≦V_(b)≦⅜, the processing contents areX, in the range of {fraction (4/8)}≦V_(a)<{fraction (6/8)} or {fraction(4/8)}≦V_(a)<{fraction (6/8)}, the processing contents are Y, and inother ranges, the processing contents are Z. Also, variousspecifications can be made for the processing contents. For example, asshown in FIG. 93 through FIG. 95, prediction taps used for processing inthe processing unit 413 can be specified with regard to pixels ofinterest.

[0779] That is, with FIG. 93, the processing contents X are illustrated,with the pixel of interest as P0, wherein filter taps P1 and P2 are setwith the pixel of interest P0 in the center in the x directionspatially, and in the same manner filter taps P3 and P4 are set with thepixel of interest P0 in the center in the y direction, and further,filter taps P5 and P6 are set before and after with the pixel ofinterest P0 in the center time-wise (e.g., tap P6 at the same pixelpositions and in one frame previous and P5 in one frame later). That isto say, the processing contents X are at a so-called space-timefiltering processing.

[0780] Also, in FIG. 94, the processing contents Y are illustrated,wherein, instead of the taps P3 and P4 in the time-space filtering inFIG. 93, a tap P12 of the timing further previous in the time-wisedirection than the tap P6, and a tap P11 of the timing further laterthan the tap P5, are set here, that is, processing contents Y areso-called time filtering processing.

[0781] Further, in FIG. 95, the processing contents Z are illustrated,wherein, instead of the taps P5 and P6 in the time-space filtering inFIG. 93, a tap P21 distanced from the pixel of interest farther than thetap P1 in the x direction, and a tap P22 distanced from the pixel ofinterest farther than the tap P2, are set. That is, the processingcontents Z are so-called space filtering processing. Note that theprocessing contents are not restricted to the three types as with a anexample shown in FIG. 92 with regard to the types thereof, and as amatter of course may be divided into a number of types other than thisnumber, for example, an arrangement may be made wherein all pixels aresubjected to binarization processing for dividing into white or black.Also, this binarization processing may be, as described in the exampleabove, for example, that which specifies binarization processing forwhether or not a pixel is to be extracted as a telop portion.Conversely, there may be three types or more for the types of processingcontents.

[0782] Next, the manual LUT changing processing in step S436 in FIG. 91will be described with reference to the flowchart in FIG. 96.

[0783] In step S441, processing contents deciding unit 1121 judgeswhether or not a pixel position is specified in processing contents arespecified as operating signals from the operating unit 1102, and theprocessing is repeated until a pixel position is specified andprocessing contents are specified as operating signals from theoperating unit 1102. That is to say, for example, in the event that thescreen shown in FIG. 97 is displayed on the display unit 403, the usercan operate the pointer 1131 on the image displayed on the display unit403 to execute predetermined processing at the pixel position to whichthe users desires to add a change in processing, displaying a drop-downlist 1132 as shown in FIG. 97, for example, and further specifying oneof the processing contents X, Y, and Z displayed in the drop-down list1132. In step S441, upon this specification being made, judgment is madethat the pixel position is specified and the processing contents arespecified, and the processing proceeds to step S442. In this case, asshown in FIG. 97, the pixel position P41 is selected, and processingcontents X are selected.

[0784] In step S442, the processing contents deciding unit 1121 readsout the corresponding combination of two types of features from thespecified pixel position. In further detail, of the features detected bythe features detecting unit 1111, the processing contents deciding unit1121 reads out the combination of the two types of featurescorresponding to the specified pixel.

[0785] In step S443, the processing contents deciding unit 1121 updatesthe processing contents matching the corresponding combination offeatures, to the processing contents specified in step S441.

[0786] In step S444, the processing contents deciding unit 1121 storesthe changed pixel position and the processing contents in the historymemory 1122.

[0787] In step S445, the processing contents deciding unit 1121 judgeswhether or not there are other changes to the LUT, and in the event thatjudgment is made that there are yet changes to be processed for the LUT,i.e., in the event that operations signals instructing other changes tothe LUT are input from the operating unit 1102, the processing returnsto step S441, and in the event that judgment is made there is noprocessing to continue changing the LUT, i.e., in the event thatoperating signals indicating ending of the changing of the LUT are inputfrom the operating unit 1102, the processing ends.

[0788] With the above manual LUT changing processing, changing of theLUT is performed as follows. That is, in step S442, in the event that,for example, (V_(a), V_(b))=(0.52, 0.27), for the features of the pixelat the obtained pixel position P41, the processing contents Y are set inFIG. 98 for the position (position from the left, position from thetop)=(5, 3) on the LUT in FIG. 98 (positions on the LUT will beindicated in the same manner hereafter, as well). In the event that theprocessing contents are changed to X in step S443 as shown in FIG. 97,the processing at the position (5, 3) on the LUT is changed from theprocessing contents Y to the processing context X in step S443, as shownin FIG. 99.

[0789] After the manual LUT changing processing shown in FIG. 96 isended, the processing returns to the processing in step FIG. 432 in theflowchart in FIG. 91, the subsequent processing is repeated. In thiscase, in the event that the LUT is a changed as described above,processing corresponding to the processing contents Y is executed up tothe pixel of the LUT wherein the combination of the two types offeatures is (5, 3), as shown in FIG. 99. Consequently, of the imageshown in FIG. 97, processing is changed corresponding to the pixelhaving the same features as the pixel position P41, i.e., the pixelbelonging to (5, 3) on the LUT, and images displayed wherein processingdifferent from that in the case of FIG. 97 is performed on that portion,as shown in FIG. 100, for example. Note that in FIG. 100, an example isshown wherein a pixel having the same features as the pixel at pixelposition P41 in FIG. 97 is white.

[0790] While description has been made regarding an example wherein(processing contents on) a LUT specified by a combination of teachersfor a selected pixel alone are changed, with the processing exampledescribed with reference to the flowchart in FIG. 96, changes to the LUTare not restricted to this. That is, for example, an arrangement may bemade wherein all nearby portions from the portion where change to theLUT has been specified on the LUT, for example, are changed processingcontents. That is, for example, as shown in FIG. 101, in the event thatall processing contents are set as Z for the initial state of the LUT(the initial state of the LUT is a so-called default setting, and may bea state wherein all processing contents are not Z), an arrangement maybe made wherein, in the event that the processing contents of theposition (4, 2) on the LUT are changed to X and the processing contentsat the position (5, 2) are changed to Y, as shown in FIG. 102A, due tothe processing of the flowchart in FIG. 96, areas near to the changedpositions on the LUT may be changed to X or Y as shown in FIG. 102B.That is, in the case of FIG. 102B, the distance from the position (4, 2)on the LUT and the distance from the position (5, 2) thereupon iscompared for all positions on the LUT, with all being changed to theprocessing contents of the position which is close. Consequently, inFIG. 102B, the left half region is made to be the processing contents X,while the right half region is made to be the processing contents Y.

[0791] In the same way, as shown in FIG. 103A, in the event that theprocessing contents of the combination the features at the positions (4,2), (7, 7) on the LUT are changed to X, and the processing contents ofthe combination the features at the positions (5, 2), (4, 5) on the LUTare changed to Y, the processing contents at (1, 1), (2, 1), (3, 1), (4,1), (1, 2), (2, 2), (3, 2), (4, 2), (1, 3), (2, 3), (3, 3), (4, 3), (5,7), (5, 8), (6, 6), (6, 7), (6, 8), (7, 5), (7, 6), (7, 7), (7, 8), (8,4), (8, 5),)8, 6), (8, 7), (8, 8), on the LUT are changed to X, as shownin FIG. 103B, and processing contents are Y at all other positions onthe LUT.

[0792] Further, in the same manner, as shown in FIG. 104A, in the eventthat the processing contents of the combination the features at thepositions (4, 2), (7, 7) on the LUT are changed to X, and the processingcontents of the combination the features at the positions (2, 3), (5,2), (4, 5), (7, 4) on the LUT are changed to Y, the processing contentsat (3, 1), (3, 2), (4, 1), (4, 2), (4, 3), (5, 7), (5, 8), (6, 6), (6,7), (6, 8), (7, 6), (7, 7), (7, 8), (8, 6), (8, 7), (8, 8), on the LUTare changed to X, as shown in FIG. 104B, and processing contents are Yat all other positions on the LUT.

[0793] Changing the LUT thus enables batch changing of the processingcontents of a combination of multiple features at a single time ofprocessing, with a relatively approximated combination of features.

[0794] Now, let us return to description of the flowchart in FIG. 91.

[0795] In step S435, and the event that judgment is made that auto LUTchanging processing as possible, the flow proceeds to step S437, andauto LUT changing processing is executed.

[0796] Now, the auto LUT changing processing will be described, withreference to the flowchart in FIG. 105. In step S461, the processingcontents deciding unit 1121 obtains a group which exists for each of theprocessing contents in the distribution of updating history stored inhistory memory 1122. That is to say, the history memory 1122 stores ahistory table which is different from the LUT, recording, for example,the position on the LUT regarding which changing has been specified bythe above-described manual LUT changing processing, and the specifiedprocessing contents, as shown in FIG. 106. The history table shown inFIG. 106 indicates that change has been instructed for the processingcontents for (3, 3), (8, 3), (2, 5), (6, 6) on the LUT, with theprocessing contents X, X, X, and Y being instructed, respectively.

[0797] Now, a group for each processing contents means an area whereinthe processing contents on the history table exist with a predetermineddensity or higher, over a predetermined area or greater.

[0798] For example, in the event that a group exists, a group 1151 isformed on the table as shown in FIG. 107, and the processing contentsdeciding unit 1121 *tames the group 1151. Note that in FIG. 107, thegroup 1151 is a group of processing contents X, and groups are obtainedfor each of other processing contents as well.

[0799] Also, a group does not exist unless manual LUT processing hasbeen executed a certain number of times, with the updating history beingstored in a history member 1122, and also with changing processing beingexecuted to the same processing contents at a position somewhat groupedtogether on the LUT. Now, in step S435 in FIG. 91, judgment is maderegarding whether or not auto LUT changing processing is possible, bywhether or not groups of the processing contents exist. That is, in theevent that a group exist for each other processing contents, judgment ismade that auto LUT changing processing is possible, and if the groups donot exist, judgment is made that auto LUT changing processing is notpossible.

[0800] In step S462, the processing contents deciding unit 1121 detectsthe position of the center of gravity of a group obtained in step S461.

[0801] That is, for example, in the case in FIG. 107, the group 1151 isformed, and position of center of gravity thereof is obtained from allpositions on the history table where processing contents X arespecified. In the case in FIG. 107, the center of gravity 1161 isobtained at all positions on the history table where processing contentsX are specified within the group 1161.

[0802] In step S463, the processing contents deciding unit 1121 convertsthe processing contents of grids on the LUT corresponding to thecombination of features of the respective pixels into processingcontents of grids on the history table corresponding to the combinationof features of pixels making up groups, corresponding to the contents ofprocessing at the position on the history table existing within apredetermined range from the position of center of gravity of the group,and the processing ends. That is to say, in FIG. 107, all the processingcontents on the LUT corresponding to the position on the history tableexist in with in the range 1162 which is the range of a circle having apredetermined radius center to on the position of center of gravity1161, are changed into processing contents X making up the group.

[0803] That is, for example, as shown in FIG. 107, in the event that thehistory table is generated in the state with an LUT configured such asshown in FIG. 108, (2, 3), (3, 4) on the history table with in the range1162 within a predetermined distance from the center of gravity 1161store history information of processing contents Y which are a differentto the processing contents X making up the group, as shown in FIG. 107,so the processing contents of (2, 3), (3, 4) on the LUT corresponding to(2, 3), (3, 4) on the history table are changed to X. Accordingly, inthe-event that LUT is configured as shown in FIG. 108, the processingcontents of (2, 3) on the LUT are Y, so the processing contents decidingunit 1121 changes the processing contents of (2, 3) on the LUT to X, asshown in FIG. 109. Also, in the case in FIG. 108, the processingcontents of (3, 4) on the LUT are X, so the processing contents decidingunit 1121 maintains this state.

[0804] Due to this processing, the processing contents on the LUT (gridinformation) is automatically changed. Note that this processing is notrestricted to being performed at the timing of the user instructingchanging processing of the LUT, and may be repeatedly executed atpredetermined time intervals.

[0805] Now, let us return to the description of the flowchart in FIG.91.

[0806] In the event that judgment is made in step S434 is that change ofthe LUT is not instructed, i.e., in the event that the user its use theimage display of the display unit 403 and judges that an image suitingthe user's preferences is generated, the processing ends.

[0807] Repeating changing of the LUT following user operations by theabove processing changes the processing contents registered to each ofthe grids on the LUT, thereby enabling images suiting the preferences ofthe user to be generated.

[0808] In the above, the processing contents on LUT are changedaccording to user operations, so it can be said that the “contents ofprocessing” are changed. Also, change is made into one or the other of:a processing system wherein the processing contents of that LUT arechanged from the position of center of gravity of the group; or aprocessing system wherein, according to processing contents at aspecified position on the LUT, the processing contents on the LUT atanother position on the LUT are changed as shown in FIG. 101 throughFIG. 104; depending on whether or not the group is obtained for eachprocessing content, according to processing performed by the user. Thatis, the changing algorithm for the LUT is changed, so the “structure ofprocessing” of the “contents of processing” can also be said to bechanged according to user operations.

[0809] Next, another configuration example of a optimizing device 1101with the processing deciding unit 1181 provided instead of theprocessing deciding unit 1112 of that optimizing device 1101 shown inFIG. 89, and an internal information generating unit 1182 newlyprovided, will be described with reference to FIG. 110. In FIG. 110, theconfiguration is the same as that of the optimizing device 1101 shown inFIG. 89 other than the processing deciding unit 1181 and internalinformation generating unit 1182 being newly provided.

[0810] The processing deciding unit 1181 stores history informationwhereby the LUT stored in the processing contents database 1191 (FIG.111) is updated, and changes the LUT corresponding to the historyinformation, while supplying the LUT stored in the processing contentsdatabase 1191 to the internal information generating unit 1182.

[0811] The internal information generating unit 1182 reads out the LUTstored in the processing contents database 1191, converts this intoinformation which can be displayed on the display unit 403, outputs tothe display unit 403, and displays.

[0812] Next, the configuration of the processing deciding unit 1181 willbe described with reference to FIG. 111. The processing deciding unit1181 in FIG. 111 is of the same configuration as the processing decidingunit 1112 except that the processing contents database 1191 is providedinstead of the processing contents database 433 shown in FIG. 90.

[0813] The processing contents database 1191 stores the LUT, and alsosupplies information of the LUT to the Internal information generatingunit 1182 low as necessary. The other functions are the same as those ofthe processing contents database 433 shown in FIG. 90.

[0814] Next, the telop extraction optimizing processing by theoptimizing device 1101 shown in FIG. 110 will be described, withreference to the flowchart shown in FIG. 112. Now, the telop extractionoptimizing processing in FIG. 112 is basically the same as theprocessing described with reference to the flowchart in FIG. 91, whereinthe processing in steps S471 through S473 and S475 through S478 in FIG.112 is processing corresponding to the steps S431 through S437 in FIG.91. Following the processing in step S473, in step S474, the internalinformation generating unit 1182 reads out the LUT of the processingcontents database 1191 of the processing deciding unit 1181, convertsthis into image signals that can be displayed on the display unit 403,outputs to the display unit 403, displays (presents) it, proceeds tostep S475, and the subsequent processing is repeated.

[0815] Due to such processing, the LUT is displayed (presented), so thatLUT can be changed while recognizing the processing performed on theimage signals which are input signals, and change of the LUT, from theimage displayed on the display unit 403.

[0816] So far, an example has been described wherein, in manual LUTchanging processing, the LUT is changed by specifying pixels on theimage processed with the processing contents registered in the LUTdisplayed on the display unit 403, and processing contents thereof, butan arrangement may be made wherein the internal information generatingunit 1182 reads out the LUT stored in the processing contents database1191 and, for example, displays the processing contents on the LUT onthe display unit 403 in a state that this directly operable by theoperating unit 1102, so that the processing contents on the LUT can bedirectly changed.

[0817] Now, manual LUT changing processing for the optimizing device1101 in FIG. 110 to directly change the values of the above-describedLUT will be described with reference to the flowchart in FIG. 113.

[0818] In step S481, the processing contents deciding unit 1121 decideswhether or not the position has been specified on the LUT, and theprocessing is repeated until specified. For example, as shown in FIG.114, in the event that the position (5, 3) on the LUT displayed on thedisplay unit 403 as a position where the processing contents are set toY is specified, judgment is made that a position has been specified onthe LUT, and the processing proceeds to step S482.

[0819] In step S482, the internal information generating unit 1182 makesa display on the display unit 403 indicating to position and on the LUTthat has been specified. That is, in the case in FIG. 114, positioneddisplayed frame 1192 is displayed at the specified (5, 3).

[0820] In step S483, the processing contents deciding unit 1121 decideswhether or not the processing contents have been specified, and repeatsthe processing until the processing contents are specified. For example,as shown in FIG. 114, specifying by a displaying a drop-down list 1193at the position where the pointer 1191 is operated (e.g., byright-clicking the mouse serving as the operating unit 1102), andselecting one of the processing contents X, Y, and Z, by the useroperating the operating unit 1102, results and judgment that processingcontents have been specified, and the processing proceeds to step S484.

[0821] In step S484, the processing contents deciding unit 1121 changesthe processing contents to the specified processing contents, and endsthe processing. That is, in the case of FIG. 114, “X” displayed in thedrop-down list 1193 is selected, so, and as shown in FIG. 115, theprocessing contents of (5, 3) on the LUT are changed from Y to X.

[0822] Due to the above processing, the processing contents set on theLUT can be directed changed, and LUT can be operated a while viewing andimage processed by the processing contents registered to the LUT, soprocessing contents suiting the preferences of the user can be readilyset.

[0823] With optimizing device 1101 in FIG. 110, the processing contentson the grids on the LUT specified by user operations are changed in themanual LUT changing processing, so it can be said that the “contents ofprocessing” are changed by user operations. Also, upon changing historystored in history memory 1122 being accumulated to a certain degree, anda group being detected, the algorithms for changing the LUT are changedfrom manual LUT changing processing to auto LUT changing processing, sothe “structure of processing” is being changed.

[0824] Further, the LUT is displayed is internal information relating toprocessing of the processing deciding unit 1112, while the processingcontents of the LUT can be changed while viewing the display LUT, so theuser can recognize the correlation relationship between the contents onthe LUT and the image displayed on the display unit 403.

[0825] Next, the configuration of an optimizing device 1201 as anotherembodiment wherein the features detecting unit 1111 and processing unit413 are provided instead of the features detecting unit 411 andprocessing unit 413 of the optimizing device 401 shown in FIG. 55, willbe described, with reference to FIG. 116.

[0826] The features detecting unit 1111 is the same as that in theconfiguration of the optimizing device 1101 in FIG. 89.

[0827] The processing unit 1211 subjects input signals read out from thebuffer 421 to mapping processing using a coefficient set obtained bylearning, for example, based on information of processing contents inputfrom the processing deciding unit 413, outputs to the display unit 403,and displays. The processing deciding unit 413 changes the method of thelearning coefficient set, based on operating signals from the operatingunit 1202. Now, operating unit 1202 is the same as the operating unit402.

[0828] Next, the configuration of the processing unit 1211 will bedescribed with reference to FIG. 117.

[0829] The learning device 1221 learns, by least-N′th-power error,coefficient sets necessary for mapping processing of the mappingprocessing unit 1222 for each of the processing contents, based on theimage signals which are input signals read out from the buffer 421 ofthe features detecting unit li1, and stores in the coefficient memory1237. Also, the learning device 1221 learns coefficient sets by changingthe value of the exponent N in least-N′th-power error, based onoperating signals input from the operating unit 1202.

[0830] The mapping processing unit 1222 reads out correspondingcoefficient sets to on the coefficient memory 1237 of the learningdevice 1221, based on the processing contents input from the processingdeciding unit 412, performs a mapping processing of the image signalswhich are input signals read out from the buffer 421 of the featuresdetecting unit 1111, outputs to the display unit 403, and displays.

[0831] Next, the detailed configuration of the learning device 1221 willbe described with reference to FIG. 118. The tutor data generating unit1231 is the same as the tutor data generating unit 231 shown in FIG. 30,which generates tutor data from input signal serving as learning data,and outputs this to the least-N′th-power error coefficient computingunit 1236. The student data generating unit 1232 is the same as thestudent data generating unit 232 in FIG. 30, which generates tutor datafrom the input signals serving as learning data, and outputs this to thefeatures extracting unit 1233 and prediction tap extracting unit 1235.

[0832] The features extracting unit 1233 is the same as the featuresextracting unit 422 of the features detecting unit 1111, which extractsfeatures from the student data and outputs these to the processingdeciding unit 1234. The processing deciding unit 1234 is the same as theprocessing deciding unit 412, which decides processing contents based onthe features input from the features detecting unit 1233, and outputsthis to the least-N′th-power error coefficient computing unit 1236. Theprediction tap extracting unit 1235 is the same as a prediction tapextracting unit 233, which, with the tutor data sequentially as pixelsof interest, extracts from the student data, pixels serving asprediction taps with regard to the pixels of interest, and outputs tothe least-N′th-power error coefficient computing unit 1236.

[0833] The least-N′th-power error coefficient computing unit 1236 is thesame as the least-N′th-power error coefficient computing unit 234 shownin FIG. 30, with regard to the basic configuration and processingcontents, which computes coefficient sets by least-N′th-power error fromthe prediction taps input from the prediction tap extracting unit 1235and the tutor data, based on information specifying the exponent Nnecessary for the least-N′th-power error coefficient computingprocessing input from the operating unit 1202, and outputs this to thecoefficient memory 1237, where it is overwritten. However, theleast-N′th-power error coefficient computing unit 1236 differs from theleast-N′th-power error coefficient computing unit 234 in FIG. 30 in thatcoefficient sets are generated for each of the processing contents inputfrom the processing deciding unit 1234. The coefficient memory 1237stores, for each of the processing contents, the coefficient sets beingoutput from the least-N′th-power error coefficient computing unit 1236for each of the processing contents. FIG. 118 illustrates coefficientsets A through N being stored for each of the processing contents.

[0834] Next, the configuration of the mapping processing unit 1222 willbe described with reference to FIG. 119. The tap extracting unit 251 isthe same as the mapping processing unit 222 in FIG. 31, which extractsprediction taps from input signal supplied from the buffer 421, withregard to the pixel of interest, and outputs to the sum of productcomputing unit 1251. The sum of product computing unit 1251 is the sameas the sum of product computing unit 252 shown in FIG. 31, which, uponinput from the prediction tap extracting unit 251, executes sum ofproduct computing processing using the value of the extracted predictiontaps (pixels) and the coefficient sets stored in the coefficient memory1237 of the learning unit 1221, generates the pixel of interest,performs this on all pixels and outputs to the display unit 403 asoutput signals, and displays. Note however, that in the sum of productcomputing processing, the sum of product computing unit 1251 uses, ofthe coefficient sets A through N stored in the coefficient memory 1237,the coefficient sets corresponding to the processing contents suppliedfrom the processing deciding unit 412.

[0835] Next, learning processing with image optimizing processing by theoptimizing device 1201 shown in FIG. 116, will be described withreference to the flowchart in FIG. 120.

[0836] In step S501, judgment is made regarding whether or not the userhas operated the operating unit 202, and in the event that judgment ismade that this has not been operated, the flow returns to step S501.Also, in the event that in step S501, judgment is made that theoperating unit 1202 has been operated, the flow proceeds to step S502.

[0837] In step S502, the tutor data generating unit 1231 of the learningdevice 1221 generates tutor data from the input signals and outputs thisto the least-N′th-power error coefficient computing unit 1236, while thestudent data generating unit 1232 and generates student data from theinput signals and outputs this to the features extracting unit 1233 andprediction tap extracting unit 1235, and the flow proceeds to step S503.

[0838] Now, as for the data used for generating the student data andtutor data (hereafter may be referred to as learning data) input signalsinput at a point in the past by a predetermined amount of time from thecurrent point, for example, may be employed. Also, for the learningdata, dedicated data may be stored beforehand, instead of using inputsignals.

[0839] In step S503, the features extracting unit 1233 extracts featuresfrom the student data at a position corresponding to the pixel ofinterest (tutor data), and outputs to the processing deciding unit 1234.

[0840] In step S504, the processing deciding unit 1234 decides theprocessing contents for the pixel of the interest based on the featuresinput from the features extracting unit 1233, and outputs this to theleast-N′th-power error coefficient computing unit 1236. For example, theprocessing deciding unit 1234 may perform vector quantizing of one ormultiple features from the features extracting unit 1233, and use thequantizing results thereof as information of the processing contents.However, in this case, there is no LUT or the like stored as with theprocessing deciding unit 1112 in FIG. 89.

[0841] In step S505, the prediction tap extracting unit 1235 takes eachpiece of tutor data as a pixel of interest, generates prediction tapsfrom the student data input from the student data generating unit 1232with regard to each of the pixel of interest, and outputs to theleast-N′th-power error coefficient computing unit 1234, and the flowproceeds to step S506.

[0842] In step S506, the least-N′th-power error coefficient computingunit 1236 judges whether or not operating signals for specifying acomputing of a coefficient set with least-N′th-power error by therecursive method (second method) have been input from the operating unit202, and in the event that judgment is made that the user has operatedthe operating unit 1202, for example, to specify not the recursivemethod, i.e., the direct method (first method), the flow proceeds tostep S507, judgment is made regarding whether coefficients a, b, and c,specifying the weight α_(s) (specifying the exponent N) in Expression(50) has been input, the processing is repeated until input, and in theevent that judgment is made that, for example, the user has operated theoperating unit 1202 and values specifying the coefficients a, b, and chave been input, the processing proceeds to step S507.

[0843] Is step S507, the least-N′th-power error coefficient computingunit 1236 solves the problem of minimizing the above-describedExpression (48) is essentially with least-square-power error, in thestate of the coefficients a, b, and c, with the weight α_(s) input,thereby obtaining the prediction coefficients w₁, w₂, w₃, . . . , w_(M),i.e., a coefficient set which is the solution for the exponent Ncorresponding to the weight α_(s), by least-N′th-power error, for eachof the processing contents input from the processing deciding unit 1234,stores these in the coefficient memory 1237, and the flow returns tostep S501.

[0844] On the other hand, in the event that a judgment is made that therecursive method is selected in step S506, the processing proceeds tostep S509.

[0845] In step S509, the least-N′th-power error coefficient computingunit 1236 judges whether or not information specifying the exponent Nhas been input, the processing is repeated until the exponent N isinput, and in the event that judgment is made to that, for example, theuser has input information's specifying the exponent N by operating theoperating unit 1202, the processing proceeds to step S510.

[0846] In step S510, the least-N′th-power error coefficient computingunit 1236 obtains a coefficient set by a solution with theleast-N′th-power error method serving as a basis. In step S511, theleast-square error coefficient computing unit 1236 obtains thecoefficient set by least-N′th-power error corresponding to the inputfrom the operating unit 1202, in a recursive matter for the exponent Nas described with reference to Expression (51) through Expression (54),using prediction values obtained from the coefficient sets obtained byleast-square-error, for each of the processing contents input from theprocessing deciding unit 1234, stores these in the coefficient memory1237, and the flow returns to step S501.

[0847] Due to the above processing, coefficient sets are learned in thecoefficient memory 1237, for each of the processing contents.

[0848] Next, the mapping processing in the image optimizing processingwith the optimizing device 1201 shown in FIG. 116 will be described,with reference to the flowchart in FIG. 121.

[0849] In step S521, the features detecting unit 1111 detects featuresof pixels of the input signals at positions corresponding to the pixelsof interest (output signals) from the image signals which are inputsignals, and outputs the detected features to the processing decidingunit 412.

[0850] In step S522, the processing deciding unit 412 decides theprocessing contents based on the features input from the featuresdetecting unit 1111, and outputs this to the processing unit 1211. Theprocessing of the processing deciding unit 412 decides the processingcontents by performing the same processing as the processing decidingunit 1234 in FIG. 118. Accordingly, as described above, in the eventthat the processing deciding unit 1234 performs vector quantizing on oneor multiple features from the features extracting unit 1233 and takesthe quantizing results thereof as information for the processingcontents, at described above, there is no LUT or the like stored in theprocessing unit 412 shown in FIG. 116, as with the processing decidingunit 1112 in FIG. 89.

[0851] In step S523, with the frame of an image serving as outputsignals corresponding to an image frame serving as current inputsignals, as a frame of interest, the tap extracting unit 251 of themapping processing unit 1222 of the processing unit 1211 takes, of thepixels of that frame of the interest, one which has not yet been madethe pixel of interest, in raster scan order, for example, as the pixelof interest, extracts a prediction tap from input signals with regard tothe pixel of interest, and outputs this to the sum of products computingunit 1251.

[0852] In step S524, the sum of products computing unit 1251 of themapping processing unit 1222 will reads out coefficient setscorresponding to the processing contents input from the processingdeciding unit 412, from the coefficient memory 1237 of the learningdevice 1221.

[0853] Then, in step S525, the sum of products computing unit 1251 usesthe prediction coefficient corresponding to the processing contents,read out from the coefficient memory 1237 of the learning device 1221,and executes sum of product computing processing with the prediction tapinput from the tap extracting unit 251 and a coefficient set read outfrom the coefficient memory 1237, following the Expression (39). Thus,the sum of products computing unit 1251 obtains the pixel value of thepixel of interest (prediction value). Subsequently, the flow proceeds tostep S526, where the tap extracting unit 251 judges whether or not allpixels in the frame of interest had been made to be the pixel ofinterest, and in the event that judgment is made otherwise, the flowreturns to step S521, a pixel which has not yet been made the pixel ofinterest is taken as a new pixel of interest in raster scan order of theframe of interest, and subsequently, the same processing is repeated.

[0854] Also, in the event that judgment is made in step S526 that allpixels in the frame of interest have been made to be the pixel ofinterest, the flow proceeds to step S527, where the display unit 403displays the frame of interest made from the pixels obtained with thesum of product computing unit 1251.

[0855] The flow then returns to step S521, or the features detectingunit 1111 detects the features of pixels of input signals at a positioncorresponding to pixel the of the interest (output signals) from theimage signals which are input signals, the next frame is taken as thenew frame of interest, and subsequently, the same processing isrepeated.

[0856] Note that with the optimizing device 1201 in FIG. 116, theexponent N is changed by the user operating the operating unit 1202 (inthe direct method, the coefficients a, b, and c specifying the exponentN are changed, while in the recursive method, the exponent N itself ischanged), thereby setting what sort of exponent N is to be employed forthe least-N′th-power error method, as a learning norm (learning system)of prediction coefficients (coefficient set). That is, the learningalgorithms for obtaining coefficients are changed. Accordingly, it canbe said that the “structure of processing” is changed so as to obtain animage suiting the preferences of the user.

[0857] Next, the configuration of the optimizing device 1301 wherein theinternal information generating unit 1312 is provided to the optimizingdevice 1201 shown in FIG. 116, will be described with reference to FIG.122.

[0858] The optimizing device 3101 in FIG. 122 is the same as theoptimizing device 1201 in FIG. 116, other than the processing unit 1311is provided instead of the processing unit 1211, by the internalinformation generating unit 1312 being provided.

[0859] The internal information generating unit 1312 reads outcoefficient set information stored for each of the processing contentsfrom the coefficient memory 1321 in the processing unit 1311, forexample, as internal information, converts this into information thatcan be displayed on the display unit 403, and displays.

[0860] Next, the configuration of the processing unit 1311 will bedescribed with reference to FIG. 123. The basic configuration is thesame, but coefficient memory 1321 is provided instead of the coefficientmemory 1237, and while the functions thereof are the same, this isconnected to the internal information generating unit 1312, with theconfiguration so that the coefficient sets stored for each of theprocessing contents are read out.

[0861] Next, the image optimizing processing with optimizing device 1301in FIG. 122 will be described, with reference to the flowchart in FIG.124. The optimizing device 1301 in FIG. 122 is also made up of learningprocessing and mapping processing, in the same way as the optimizingdevice 1201 in FIG. 116. With the learning processing, processing thesame as steps S501 through S511 in FIG. 120 are performed in steps S541through S551.

[0862] Further, with the learning processing, following the processingin steps S541 and S551, the flow proceeds to step S552, the internalinformation generating unit 1312 reads the coefficient sets stored inthe coefficient memory 1321 as internal information, and generates imagesignals that can be displayed based on the values contained in thecoefficient sets, outputs to the display unit 403, and displays.

[0863] At this time, the image generated by the internal informationgenerating unit 1312 and displayed on the display unit 403 may have aform of a 3-dimensional distribution such as shown in FIG. 39, or2-dimensional distribution such as shown in FIG. 40, as described above.

[0864] Now, let us return to the description of the flowchart in FIG.124.

[0865] Following the processing in step S552, the flow returns to stepS541, and subsequently, the processing is repeated.

[0866] On the other hand, with the mapping processing shown in theflowchart in FIG. 125, in steps S571 through S577, processing the sameas that in steps S521 through S527 in FIG. 121 is performed,respectively.

[0867] Due to the above processing, the values of coefficient sets (thecoefficient values) stored in the coefficient memory 1321 of theprocessing unit 1311 art display (presented) as internal informationrelating to processing, so the user operates the operating unit 1202 toso as to obtain an image as output signals suiting his/her ownpreferences while viewing the coefficient set distribution andprocessing results of the processing unit 1311 as output signals,thereby changing the learning algorithm for obtaining coefficient setsby changing the exponent N. Accordingly, it can be said that the“structure of processing” is being changed so as to obtain an imagesuiting the preferences of the user. Also, while the coefficient setsare displayed in the above example, an arrangement may be made whereininternal information relating to the processing, such as for example,whether the current least-N′th-power error method is the direct methodor recursive method, is displayed.

[0868] Next, an optimizing device 1401 having a configuration providedwith a processing unit 1411 instead of the processing unit 1211 of theoptimizing device 1201 in FIG. 116, will be described with reference toFIG. 126.

[0869] The configuration of the processing unit 1411 is basically thesame as the configuration of the processing unit 311 of the optimizingdevice 301 in FIG. 41, which optimize is input signals based on theoperating signals input from the operating unit 1202 and the processingcontents input from the processing deciding unit 412, and displays theseon the display unit 403.

[0870] Next, the configuration of the processing unit 1411 will bedescribed with reference to FIG. 127. Multiple coefficient sets arestored for each of the processing contents in the coefficient memory1421, with coefficient sets necessary for the mapping processing by themapping processing unit 1222 being stored. The figure indicates thecoefficient sets A through N are stored. These coefficient sets aregenerated beforehand by learning, with the learning device 1441 in FIG.128.

[0871] Now, the configuration of the learning device 1441 generatingthese coefficient sets will be described with reference to FIG. 128.

[0872] The tutor data generating unit 1451, student data generating unit1452, features extracting unit 1453, processing deciding unit 1454, andprediction tap extracting unit 1455, of the learning device 1441,correspond to the tutor data generating unit 1231, student datagenerating unit 1232, features extracting unit 1233, processing decidingunit 1234, and prediction tap extracting unit 1235, of the learningdevice 1221 shown in FIG. 118, respectively, and accordingly descriptionthereof will be omitted.

[0873] The normal equation generating unit 1456 is the same as thenormal equation generating unit 354 shown in FIG. 43, which generates anormal equation based on tutor data input from the tutor data generatingunit 351 and prediction taps, and outputs to the coefficient decidingunit 1457, but differs in that, at this time, generates and outputsnormal equations for each of the pieces of processing contentsinformation input from the processing deciding unit 1454.

[0874] The coefficient deciding unit 1457 is the same as the coefficientdeciding unit 355 in FIG. 43, which solves the input normal equation isto generate coefficient sets, and at this time it generates coefficientsets correlated with each piece of processing contents information.

[0875] Next, the coefficient deciding processing (learning processing)with the learning device 1441 in FIG. 128 will be described, withreference to the flowchart in FIG. 129. In step S591, the tutor datagenerating unit 1451 generates tutor data from learning data, andoutputs this to the normal equation generating unit 1456, while thestudent data generating unit 1452 generates student data from learningdata, and outputs this to the features extracting unit 1453 and theprediction tap extracting unit 1455, and the flow proceeds to step S592.

[0876] In step S592, the prediction tap extracting unit 1455 extractsprediction taps from student data with regard to each data of interest,taking each piece of tutor data sequentially as a pixel of interest, andoutputs this to the normal equation generating unit 1456, and the flowproceeds to step S593.

[0877] In step S593, the features extracting unit 1453 extracts featuresregarding the student data (pixel) at the position corresponding to thepixel of interest (tutor data), and outputs this to the processingdeciding unit 1454.

[0878] In step S594, the processing deciding unit 1454 decides theprocessing contents for each pixel, based on the features extracting itby the features extracting unit 1453, and outputs decided processingcontents information to the normal equation generating unit 1456. Forexample, the processing deciding unit 1454 may subject one or multiplefeatures to vector quantizing, and take the quantizing results as theinformation for processing contents. Accordingly, there is no LUT storedin the processing deciding unit 1456.

[0879] In step S595, the normal equation generating unit 1456 uses thetutor data and prediction taps sets to compute this summation (1) of thecomponents of the matrix at the left-hand side of the Expression (46)and the summation (Σ) of the components of the vector of the right-handside thereof, thereby generating the normal equation which is input fromthe processing deciding unit 1454 for each piece of processing contentsinformation, and is output to the coefficient deciding unit 1457.

[0880] The flow then proceeds to step S596, where the coefficientdeciding unit 1457 solves the normal equation input from the normalequation generating unit 1456 for each piece of processing contentsinformation, and obtains a coefficient set by so-called least-squareerror for each piece of processing contents information, which is storedin the coefficient memory 1421 in step S597.

[0881] Due to the above processing, a basic coefficient set (coefficientset serving as an initial value) is stored in the coefficient memory1421 for each piece of processing contents information. Now, while theabove description gives the coefficient set being obtained byleast-square error, this may be a coefficient set obtained by a anothermethod, and may be a coefficient set obtained by the above-describedleast-N′th-power method.

[0882] Next, the image optimizing processing of the optimizing device1401 shown in FIG. 126 will be described with reference to the flowchartin FIG. 130. This image optimizing processing is made up of coefficientchanging processing and mapping processing, but the mapping processingis the same as the mapping processing described with FIG. 121 and FIG.125, so only the coefficient changing processing will be described here.

[0883] In step S611, the changing processing unit 332 of the coefficientchanging unit 322 judges whether or not operating signals for operatingthe coefficient values have been input from the operating unit 1202.That is, in the event that the user views the image displayed on thedisplay unit 403 and deems this to be suiting his/her own preferences,the mapping processing will be executed coefficient sets for each pieceof processing contents information stored in the coefficient memory1421, but in the event that this does not suit the preferences of theuser, operations for changing the coefficient set stored in thecoefficient memory 1421 used for mapping processing, is performed.

[0884] For example, in step S611, in the event that judgment is madethat operating signals for operating coefficients have been input, i.e.,in the event that the operating unit 1202 has been operated so as tochange one of the coefficient values of the coefficients stored in thecoefficient memory 1421, the flow proceeds to step S612.

[0885] In step S612, the changing processing unit 332 controls thecoefficient read/write unit 331 to read out the coefficient sets storedin the coefficient memory 321, and the flow proceeds to step S613. Instep S613, the changing processing unit 332 makes judgment regardingwhether or not the change of the coefficient value input as an operatingsignal is equal to or more than a predetermined threshold value S11 incomparison with the value contained in the coefficient set beforehand.In the event that, for example, judgement is made in step S613 that thechange of the coefficient value input as an operating signal is equal toor more than the threshold value S11 as to the coefficient set valuestored in the coefficient memory 1421, the processing proceeds to stepS614.

[0886] In step S614, the changing processing unit 332 changes the valuesof the coefficients contained in the coefficient set with the springmodel as shown in FIG. 50, and the processing proceeds to step S615.

[0887] On the other hand, in the event that judgment is made in stepS613 that the changed between the value input as an operating signalthat the value of the coefficients set stored in the coefficient memory1421 is not equal to or more than the threshold value S11, theprocessing proceeds to step S615.

[0888] In step S615, the changing processing unit 332 changes the valueof the coefficients contained in the coefficient set with theequilibrium model such as shown in FIG. 51, and the processing proceedsto step S616.

[0889] In step S616, the changing processing unit 332 controls thecoefficients read/write unit 331 to store the changed coefficient setvalues in the coefficient memory 1421 by overwriting, the processingreturns to step S611, and the subsequent processing is repeated.

[0890] Then, in the event that judgment is made in step S611 that thecoefficient values have not been operated, i.e., in the event that theuser judges that the image displayed on the display unit 403 is an imagesuiting the preferences of the user, the flow returns to step S611, andsubsequently, the same processing is repeated.

[0891] Due to the above coefficient changing processing, the user canchange coefficient sets and stored for each piece of processing contentsinformation used for mapping processing, and executes processing that isoptimal for the user. Now, changing the values of the coefficients in acoefficient set means that the “contents of processing” of the mappingprocessing with the mapping processing unit 311 are being changed.

[0892] Also, with the coefficient changing processing in FIG. 130, inthe event that the magnitude of change of the coefficients is equal toor greater than a predetermined threshold value S11, all coefficientvalues in the coefficient set are changed with the spring model,according to the value of the coefficients operated, and in the eventthat this is smaller than the threshold value S11, all coefficientvalues in the coefficient set are changed with the equilibrium model, sothe coefficient set changing algorithm for changing the coefficients setchanges. Accordingly, with the processing unit 1411 of the optimizingdevice 1401 in FIG. 126 as well, it can be said that the “contents ofprocessing”, and further, the “structure of processing” is also changed,following user operations, whereby signal processing that is optimal forthe user is being performed.

[0893] Also, in the event that the coefficient set stored in thecoefficient memory 1421 is obtained by least-N′th-power error, asdescribed above, an arrangement may be made wherein, for example,coefficient sets corresponding to multiple exponents N are stored in thecoefficient memory 1421 beforehand, and the coefficient changing unit322 changes to a coefficient set corresponding to the specified exponentN, following the operating signals from the operating unit 1202 based onoperations made by the user. In this case, the coefficient sets storedin the coefficient memory 1421 are changed to those generated byleast-N′th-power error corresponding to the exponent N input from theoperating unit 1202, based on user operations. That is, the learningalgorithms for obtaining the coefficient sets corresponding to exponentsN changed, so it can be said that the “structure of processing” is beingchanged.

[0894] Next, the configuration of an optimizing device 1501 wherein aninternal information generating unit 1521 is provided to the optimizingdevice 1401 shown in FIG. 126, will be described with reference to FIG.131. With the optimizing device 1501 in FIG. 131, the otherconfigurations are the same as the optimizing processing unit 1401 inFIG. 126 other than an internal information generating unit 1521 beingprovided, and the processing unit 1511 being provided instead of theprocessing unit 1411.

[0895] The internal information generating unit 1521 reads outcoefficient sets stored for each piece of processing contentsinformation stored in the coefficient memory 1531 of the processing unit1511 for example, as internal information, converts this into imagesignals which can be displayed on the display unit 403, outputs to thedisplay unit 403, and displays.

[0896] Next, the configuration of the processing unit 1521 will bedescribed with reference to FIG. 132. This is basically the same as theconfiguration of the processing unit 1411 in FIG. 126, except thatcoefficient memory 1531 is provided instead of the coefficient memory1421. The coefficient memory 1531 is functionally almost the same as thecoefficient memory 1421, but is different in that it is connected to theinternal information generating unit 1521, and is of a configurationwherein coefficient sets are read out as appropriate.

[0897] Next, image optimizing processing with the optimizing device 1501shown in FIG. 131 will be described with reference to the flowchart inFIG. 133. As with the image optimizing processing performed by theoptimizing processing 1401 shown in FIG. 126, this image optimizingprocessing is made up of coefficient changing processing and mappingprocessing, but the mapping processing is the same as the mappingprocessing described with FIG. 121 and FIG. 125, so only the coefficientchanging processing will be described here.

[0898] With the coefficient changing processing, processing the same asthat in steps S611 through S616 in FIG. 130 is performed in steps S631through S636, respectively.

[0899] Then, in step S636, as with the case in step S636 in FIG. 130,following the post-change coefficient sets being stored in thecoefficient memory 1521, the flow proceeds to step S637, where theinternal information generating unit 1521 reads out the coefficient ofvalues of the coefficient set stored in the coefficient memory 1531,converts these into image signals which can be displayed on the displayunit 403, outputs to the display unit 403, and displays (presents). Atthis time, the coefficient values of the coefficient set displayed(presented) on the display unit 403 may have, for example, a form of a3-dimensional distribution such as shown in FIG. 39, or 2-dimensionaldistribution such as shown in FIG. 40, as described above.

[0900] Following the processing in step S637, the flow returns to stepS631, and subsequently, the same processing is repeated.

[0901] According to the coefficient changing processing in FIG. 133, thevalues of the coefficient set stored in the coefficient memory 1531 foreach piece of processing contents information are displayed as internalinformation, so the user can operate the operating unit 1202 whileviewing these coefficient sets, so as to obtain coefficient sets whichexecute processing that is optimal for the user.

[0902] Also, with the sum of product computing unit 1251 of the mappingprocessing unit 1222, output signals can be obtained by a computing ahigher order expression of an order of two or higher, instead of thelinear expression in Expression (39).

[0903] Next, the above-described series of processing may be carried outby hardware, or may be carried out by software. In the event that theseries of processing is carried out by software, a program making up thesoftware is installed in a general-purpose computer or the like.

[0904] Now, FIG. 134 illustrates the configuration example of the firstembodiment of a computer to which a program for executing theabove-described series of processing is installed.

[0905] The program can be stored in a hard disk 2105 or ROM 2103 servingas recording medium built into the computer, beforehand.

[0906] Or, the program may be temporarily or permanently stored(recorded) on removable recording media 2111, such as a floppy disk,CD-ROM (Compact Disc Read-Only Memory), MO (magneto-optical) disk, DVD(Digital Versatile Disk), magnetic disk, semiconductor memory, etc. Sucha removable recording medium 2111 can be provided as so-called packagedsoftware.

[0907] Also, besides being installed from such a removable recordingmedium 2111 to a computer, the program may be transferred from adownload site to the computer by wireless means via a satellite fordigital satellite broadcasting, or by cable via a network such as a LAN(Local Area Network) or the Internet, with the computer receiving theprogram transferred thus with a communication unit 2108 and installingthe program in a built-in hard disk 2105.

[0908] The computer has a built-in CPU (Central Processing Unit) 2102.An input/output interface 2110 is connected to the CPU 2102 via a bus2101, and upon commands being input via the input/output interface 2110by an input unit 2107 configured of a keyboard, mouse, microphone, etc.,operated or the like by a user, the CPU 2102 executes the program storedin the ROM (Read Only Memory) 2103 accordingly. Or, the CPU 2102 loadsto RAM (Random Access Memory) 2104 and executes a program stored in thehard disk 2105, a program transferred from a satellite or network andreceived with the communication unit 2108 and installed in the hard disk2105, or a program read out from a removable recording medium 2111mounted to a drive 2109 and installed to the hard disk 2105. Thus, theCPU 2102 performs the processing following the above-describedflowchart, or the processing carried out by the configuration of theabove-described block diagrams. The CPU 2102 outputs the processingresults from an output unit 2106 configured of an LCD (Liquid CrystalDisplay) or speakers or the like, via the input/output interface 2110,as necessary, for example, or transmits from the communication unit2108, and further performs recording or the like to the hard disk 2105.

[0909] Now, in the specification, the processing steps describing theprogram for causing the computer to perform various types of processingdo not necessarily need to be performed in a time-sequence following thedescribed order in the flowcharts, and the present invention includesexecuting in parallel or independent processing (e.g., parallelprocessing or processing by objects).

[0910] Also, the program may be processed by one computer, or may bedivided among and processed by multiple computers. Further, the programmay be transferred to a remote computer to be executed.

[0911] While the above has been the description of applying the presentinvention to noise removal from input signals or auto-driving, thepresent invention is broadly applicable to applications other than noiseremoval or auto-driving, e.g., conversion of frequency properties ofsignals, and so forth.

INDUSTRIAL APPLICABILITY

[0912] As described above, according to the present invention, thecontents of processing, and further, the structure of processing ischanged based on user operations, so processing which is optimal forthat user can be performed.

[0913] With the present embodiment, internal information is displayed onthe same display unit as output signals, but internal information may bedisplayed on a display unit separate from the display unit fordisplaying output signals.

1. A signal processing device, comprising: signal processing means for signal processing of input signals; and output means for outputting the signal processing results of said signal processing means; wherein the structure of processing of said signal processing means is changed based on operating signals supplied according to user operations.
 2. A signal processing device according to claim 1, said signals processing means comprising: characteristics detecting means for detecting characteristics from said input signals; processing deciding means for deciding contents of processing as to said input signals, based on characteristics detected by said characteristics detecting means; and processing executing means for executing processing to said input signals, following said contents of processing decided by said processing deciding means; wherein the structure of processing of at least one of said characteristics detecting means, said processing deciding means, or said processing executing means, is changed based on said operating signals.
 3. A signal processing device according to claim 1, wherein said output means comprise presenting means for presenting the signal processing results of said signals processing means.
 4. A signal processing device according to claim 2, wherein the structure of processing of said characteristics detecting means is changed based on said operating signals.
 5. A signal processing device according to claim 4, wherein said operating signals are signals which specify, of a plurality of types of characteristics, a predetermined number of types of characteristics; and wherein said characteristics detecting means change the structure of processing thereof so as to detect said predetermined number of types of characteristics.
 6. A signal processing device according to claim 5, wherein said characteristics detecting means detect said predetermined number of types of characteristics from said input signals; and wherein said processing deciding means decide the contents of processing at said processing executing means as to said input signals, based on said predetermined number of types of characteristics detected from said input signals by said characteristics detecting means.
 7. A signal processing device according to claim 6, wherein said input signals are image signals; and wherein said processing deciding means decide whether or not to output said input signals without change as said contents of processing as to said input signals, based on said predetermined number of types of characteristics detected from said input signals by said characteristics detecting means; and wherein said processing executing means detect a telop within image signals which are said input signals, by selectively outputting said input signals following the decision of said processing deciding means.
 8. A signal processing device according to claim 4, wherein said characteristics detecting means change the structure of processing thereof so as to detect new types of characteristics differing from characteristics which are provided beforehand, based on said operating signals.
 9. A signal processing device according to claim 8, wherein said characteristics detecting means detect the dynamic range of said characteristics prepared beforehand, maximum value, central value, minimum value, sum, dispersion, the number of input signals with said characteristics having a value greater than a threshold value, or and linear combination of said characteristics, as said new types of characteristics.
 10. A signal processing device according to claim 2, wherein the structure of processing of said processing deciding means are changed based on said operating signals.
 11. A signal processing device according to claim 11, wherein said processing deciding means store a characteristics/processing correlation relationship which is a correlation relationship between the values of said characteristics, and said contents of processing as to said input signals having characteristics of those values; and decide the contents of processing correlated to the values of characteristics detected from said input signals as the contents of processing as to the input signals, in said characteristics/processing correlation relationship.
 12. A signal processing device according to claim 11, wherein said processing deciding means change the structure of processing thereof by changing said contents of processing in said characteristics/processing correlation relationship, based on said operating signals.
 13. A signal processing device according to claim 11, wherein, in said characteristics/processing correlation relationship, two contents of processing exist as to said input signals; processing for outputting output signals of a first value, and processing for outputting output signals of a second value; and wherein said processing executing means binarize said input signals into said first and second values, following the decision of said processing deciding means.
 14. A signal processing device according to claim 2, wherein the structure of processing of said processing executing means is changed based on said operating signals.
 15. A signal processing device according to claim 14, said processing executing means comprising: tutor data generating means for generating tutor data from predetermined learning data; student data generating means for generating student data from said learning data; learning means for learning a prediction value of said tutor data obtained by linear combination of said student data and a predetermined prediction coefficient, and said prediction coefficient which minimizes the error with the tutor data; and output signals generating means for generating output signals by performing linear combination of said input signals and a prediction coefficient obtained by said learning means.
 16. A signal processing device according to claim 15, wherein said learning means learn said prediction coefficient by least-N′th-power error which statistically minimizes an N′th-power error which is said error to the N′th power; and change the structure of processing thereof by changing the N′th power of said error, based on said operating signals.
 17. A signal processing device according to claim 16, wherein said learning means change said N′th power of error based on said learning signals, by employing the product of a square-error as said N′th power of error and a weight according to said operating signals.
 18. A signal processing device according to claim 16, wherein said learning means employ, as said N′th power error using an N′th power corresponding to said operating signals, the product of a square-error and a N−2′th error of a prediction value of said tutor data computed using said prediction coefficient obtained by least-N′th-error.
 19. A signal processing device according to claim 15, wherein said learning means learn said prediction coefficient for each contents of processing which said processing deciding means decide; and wherein said output generating means generate said output signals by linear combination of said input signals, and said prediction coefficient corresponding to said contents of processing which said processing deciding means decide based on said input signals.
 20. A signal processing device according to claim 1, said signals processing means comprising: judging means for monitoring said operating signals judging whether or not usable for learning; learning means for learning a correction norm which is a norm for correcting said input signals, based on learning operating signals which are said operating signals usable for learning; and correcting means for correcting said input signals based on said correction norm obtained by said learning, and outputting the post-correction signals thereof as output signals.
 21. A signal processing device according to claim 1, said signals processing means comprising: tutor data generating means for generating tutor data from predetermined learning data; student data generating means for generating student data from said learning data; learning means for learning a prediction value of said tutor data obtained by linear combination of said student data and a predetermined prediction coefficient, and said prediction coefficient which minimizes the error with the tutor data; and output signals generating means for generating output signals by performing linear combination of said input signals and a prediction coefficient obtained by said learning means.
 22. A signal processing device according to claim 1, wherein said signal processing means learn said prediction coefficient by least-N′th-power error which statistically minimizes an N′th-power error which is said error to the N′th power; and change the structure of processing thereof by changing the N′th power of said error, based on said operating signals.
 23. A signal processing device according to claim 22, wherein said learning means change said N′th power of error based on said operating signals, by employing the product of a square-error as said N′th power of error and a weight according to said operating signals.
 24. A signal processing device according to claim 22, wherein said learning means employ, as said N′th power error using an N′th power corresponding to said operating signals, the product of a square-error and a N−2′th error of a prediction value of said tutor data computed using said prediction coefficient obtained by least-N′th-error.
 25. A signal processing method, comprising: a signal processing step for signals processing of input signals; and an output step for outputting the signal processing results of the processing in said signal processing step; wherein the structure of processing in said signal processing step is changed based on operating signals supplied according to user operations.
 26. A recording medium storing a computer-readable program, said program comprising: a signal processing control step for controlling signal processing of input signals; and an output control step for controlling output of the signal processing results of the processing in said signal processing control step; wherein the structure of processing in said signal processing control step is changed based on operating signals supplied according to user operations.
 27. A program, causing a computer to execute: a signal processing control step for controlling signal processing of input signals; and an output control step for controlling output of the signal processing results of the processing in said signal processing control step; wherein the structure of processing in said signal processing control step is changed based on operating signals supplied according to user operations. 